PL7: I-O, I-O, so off to work I go

I've said this before, but it bears repeating:  People buy software to do something to data.  For a more extensive discussion of this, see the Internationalization Myth #1 blog.  Go on and read it, it's very short, and I can wait.

There, that didn't take long.  And now my meaning is clear, you can see how important it is to test software to find out whether it can handle data in a particular character encoding.  But the problem is, where can you get the encoded data to run through the system in a hurry?  Easy, just take your test data files, of which you most certainly have an extensive collection, and pseudo-localize them.  You might need to tweak your pseudo-localization tools a bit, but it's well worth it.

If the data is pseudo-localized with characters from a broad cross-section of Unicode, you'll be able to see if Unicode is processed and output correctly, without mangling anything.  Using other character sets, such as EUC-JP, will bear out whether your software can handle those as well.  It doesn't matter if it's supposed to convert, reject, or process these character sets directly, so long as it handles them in an appropriate manner for the markets you want to sell to.

Check the output and make sure the data has not been corrupted.  For example, if the phrase "Some pseudo localized text" were pseudo-localized with a cross-section of Unicode characters, it might look like this:

§õʍϵ Рѕәטᴆᴑ ᴌᴼᶜᶏḼἷ₹ﭺ口 下王丈৳

Just in case it doesn't get to you unmangled via this blog, or you don't have the fonts loaded, here is an image:

If the software you're testing is supposed to store it in a database, and then retrieve it intact, make sure that the output exactly matches the input.  If it's supposed to break the text into individual characters, check that the characters are the same.  You get the idea.  With pseudo-localization, it should be easy to automate the testing (provided the testing software is properly internationalized) as well as visually test it.

Please don't consider this a substitute for proper multilingual testing.  Each language has its own formatting and processing rules that cannot be tested via a simple pseudo-localization.  The classic example is Thai, which has no spaces or syntactic characters to divide words.  In order to parse Thai words correctly, a dictionary must be used, along with some grammatical processing.  In correct pro cess in go pens up big can sofw. or Ms.  Forth E sake of you ruse RS, create some actual input test data from the languages used in your markets.

And remember that this does not test locale-specific data handling.  That data will also have to be created to represent locales in your target markets.

Test the I-O, your 亹stomers wiѭ th⌘k yo=FA for ☃.  Or at least they'll be able to.

PL6: Don't get too attached to fragments

While fragments might be good for a Greek mosaic, they aren't useful for Greek translation.  Or any other translation, for that matter.

In moving text strings to external resources, some engineers try to be as efficient as possible. Efficiency is certainly a laudable attribute, but when it comes to strings, it may mean taking a less obvious path.

Let me clarify.  A simple set of user interface strings might be:

n file(s) have been deleted.
n folder(s) have been deleted.
n file(s) have been moved.
n folder(s) have been moved.

Looking at these strings in English, one might be tempted to break them down into what seem to be obvious components:

file(s)
folder(s)
have been deleted.
have been moved.

Super!  The code just needs to pick up the components and concatenate them. Fewer words, less storage space taken, and the translation should be cheaper and faster, right?

Well, no, not exactly.  The problem is that while in English these components can be simply concatenated and still make sense, this is not true in other languages.  Possible strings appearing in the UI in somewhat stilted English as:

1 file(s) have been deleted.
2 file(s) have been deleted.
1 folder(s) have been deleted.
2 folder(s) have been deleted.
1 file(s) have been moved.
2 file(s) have been moved.
1 folder(s) have been moved.
2 folder(s) have been moved.

Should become in Italian:

1 file è stato eliminato.
2 file sono stati eliminati.
1 cartella è stata eliminata.
2 cartelle sono state eliminate.
1 file è stato spostato.
2 file sono stati spostati.
1 cartella è stata spostata.
2 cartelle sono state spostate.

Ignoring the possibility that this might not be the best Italian translation, note how many changes occur from sentence to sentence, due to changes in gender and number. The translator is not given the option to accommodate these changes, and must cope. The resulting translation would be extremely poor, akin to English users seeing a string such as "These file she have been deleted." Not exactly user-friendly.

Using a pseudo-localization string that has some sort of opening and closing characters, such as curly braces, will expose concatenated fragments. If a closing brace is followed by an opening brace, chances are the resource strings need some defragging.

PL5: Expanding your universe

I expect this next test in the series doesn't cover the universe, but it will get you further towards whole Earth applications.

The pseudo-localization applied to your text resources should grow the length of the strings. Depending on your application, I recommend the length is increased by about 50%, at least for short strings. The additional text should also include Asian characters, that is, characters from the Han range of Unicode. Let me explain why.

One aspect of testing is edge cases - testing limits and restrictions to see if they are working and how. This includes minimum, below minimum, maximum, and above maximum lengths. Translation usually grows the length of strings, both in byte count and in display. Fields in the user interface (UI) are often limited in length (byte and display), due to display constraints. Expanded pseudo-localized text will show how the UI could change. And more.

What are you looking for? For starters, you're looking at the layout and aesthetics of the UI with the expanded text. Is text wrapping awkwardly? Is it overlapping other text or screen elements? Is it pushing other objects out of position, skewing the overall layout? Do items line up where they're supposed to? Is the text truncated? Obviously limitations are necessary due to screen size, especially for mobile applications. But translators aren't mind readers - they're probably only seeing text in a resource file or translation tool. The only indication they'll have of a length restriction is the comment that you write next to the string to be translated. Which of course you have done.

Remember that I recommended using Asian text in the pseudo-l10n? This is useful for checking that these characters are legible in the space provided; by legible I mean that the stroke lines are distinct and the tops or bottoms of the character are not cut off. Asian characters are often more intricate and complex, requiring a larger font and additional vertical space to be rendered legibly. However, Asian translations will frequently shrink in width. A pseudo-l10n won't bear that out because it usually adds on to the existing string length. But you could run one that replaces the string with shorter Asian character strings, and see what that does to the UI.

PL4: For display only

This next pseudo-localization test in the series is very straightforward. Maybe.

You can verify whether your software correctly handles a particular character encoding by pseudo-localizing the resource files into that encoding. Use a broad spectrum of characters from the encoding, if not all of them, to check that the entire set is handled properly. Simply pseudo-localize the resources, bring up the pseudo-localized user interface, and view the text. Of course, you have to know what you're looking at, but if you've familiarized yourself with the set of characters used in the pseudo-localization, that shouldn't be a problem.

But that's not all! Since all of you are using a Unicode encoding for your user interface (a-hem), there's something else you can test. Some characters use a different font depending on the language they are representing. So for example, if characters from the Han section of Unicode are used in your pseudo-localization, you can set the locale to a Chinese region and verify the characters are in a Chinese font. Then change the locale to a Japanese region and see if a Japanese font is used to display them. Again, you have to know what you're looking at, but if your software is responsible for display, this is an important test. (Take a look at these charts, particularly the second and third sections.) If necessary, select a few key Han characters that differ in Chinese and Japanese, make up an image chart, and familiarize yourself with them. As they're used over and over in the pseudo-localization, you'll get to know them well.

Your customers will be happy you did.

PL3: May we dance with your dates?

This is the third entry in my series of the 9 coding areas pseudo-localization can test.

Amongst ourselves in internationalization, we refer to certain types of data as locale-sensitive. This is data that changes in some way from region to region (locale to locale), typically in format. The classic example is a date. And the classic example of this example is the short format in the US, month/day/year, that becomes day/month/year in most European countries; today is January 11, 2011, written 1/11/11 in the US, but 11/1/11 in the UK. (Often this is accompanied by some US baiting about how illogical that format is, but we won't go there and mention that in human terms, the day becomes less important beyond a week or so, and the month comes into prominence, lasting an entire year, when finally the year is needed, so it's stuck on the end. And it's written month day, year in longer formats, as above. But I won't mention any of that here.)

There are many more locale-sensitive pieces of data: numbers, time, prices, measurements, weights, telephone numbers, addresses, sizes, etc. Some of these data formats have been standardized, so that programs can select them using locale identifiers and apply them to the data. The Common Locale Data Repository (CLDR) is a public database with locale formats for many locales throughout the world (though admittedly there are no sizes, nor telephone numbers, nor postal addresses). Internationalization library functions and methods use it for formatting and parsing locale-sensitive data.

As these formats can be accessed programmatically for most of the locales of the world, they should be. That is, rather than externalize a date format for a localizer to alter to suit another locale, dates should be programmatically formatted. Why? The long answer is another blog entry, however the short answer is that there are many more formats than there are localizations. For example, the English localization is usually a single localization, but formats for English-speaking locales vary (see the short date format example above). The same is true for French, Spanish, Traditional Chinese, and so on.

But what does all this have to do with pseudo-localization testing?

You may remember my post about using pseudo-l10n to test whether localized files are picked up when they should be. Or perhaps you don't, in which case you might want to review it. Anyway, once you've set up your system such that the pseudo-localized files are mimicking an actual locale, you can exploit that setup to check on whether locale-sensitive formats are programmatically determined.

Run the system with the locale set to the pseudo-localized files' locale, and pay special attention to the formats of dates and numbers. Now change the locale setting to one that has different formats and run again. Check the dates and numbers - have they changed format? You might want to verify the formats against those listed in the CLDR; if you're looking online, check the summary charts. Start with the language base link, then drill down to get to the specific locale you're looking for.

Change the locale to a third value and check again. The formats should be changing appropriately each time. If not, to your customers today might just look like November 1, 2011.

PL2: It's good to be resourceful

Another aspect of software internationalization that pseudo-localization can test is whether all relevant resources have been made localizable.

Now, I understand that in order for this test to work, everything that is shown to the user must be visually inspected. This means the entire fixed user interface must be made to display, including windows that come up when various menu options are selected, pop-up windows, help messages, alternate text, user messages, and, that most difficult of entities to bring to view, error messages.

It's funny, when I tell people pseudo-localization can test 9 aspects of internationalization (actually I started with 5 and found more as time went on), this is the test they claim is not valid. And yet, at the same time, this aspect is the primary one that people claim they are testing when they pseudo-localize their product. Go figure. Personally I'm not confident that you can verify the entire user interface has been properly externalized this way, only a good portion of it, and therefore likely all of it. The key word is likely.

So run your pseudo-localization tool, bring up the product (hopefully it picks up the pseudo-localized resources - see the previous post), and have a good look. Try to create some errors so you can at least check that some error messages are being picked up correctly. In case you're wondering, you're checking for text that is still in the source language, without the additional characters or character transformations that the pseudo-l10n creates.

And then, move on to the next test.

PL1: A loose l10n is an easy pick-up

The first in my series of the 9 coding areas pseudo-localization can test.
Note: localization is often abbreviated l10n, because there are 10 letters between the l and the n. Sometimes people will capitalize the L to distinguish it from the 1. (PL1 is Pseudo-Localization 1, not Programming Language 1, for those of you old enough to remember that.)

Providing the pseudo-l10n is configured as a real localization will be, it tests whether or not the product will pick up the translated resources.

Once you run your resource files through a pseudo-translator tool, set them up as being for some locale other than the source locale. For example, if your code is typically written in the US and run on environments set to the en-US language/locale, then treat your pseudo-localized files as if they were the Japanese translation for Japan, that is, ja-JP. That typically means appending "ja-JP" to the file names, but may mean they should be stored in a ja-JP directory or database. This exercise also provides familiarity with the localization structure of the product - where the resources reside and what the naming conventions are.

Next, continuing with the example, the environment needs to look like a typical Japanese customer's. Whether that means the system settings are set to ja-JP, or the software itself is set to Japanese, or just the session, make it so. Maybe any of those settings are supposed to make your software switch to Japanese - all environments should then be tested. A side benefit of this is learning how other language/locale environments are set, and how the product is supposed to behave (yes, many folks don't know what the expected behavior should be in some cases).

Now, run your software. Is it picking up the pseudo-Japanese files in all areas of the product? Is it displaying date, numeric, and other dynamically determined formats according to the Japanese locale preferences? Did it do this without requiring a rebuild?

If not, you've got some work to do.

Long time, new series - The 9 Coding Areas Pseudo Localization can Test

Hello out there!
It's been a very long time, but not in my head. That is, I always have lots of topics that I'd like to blog about, just very little time to blog. The topic I'd like to begin on today is pseudo-localization, also known as pseudo-translation, pseudolocalization, pseudo-l10n, and probably some other names.

What pseudo-localization is
Pseudo-localization is the programmatic localization of a software product to enable early testing of product localizability. Typically this entails automatically transforming the translatable text into characters outside of ASCII in another character encoding. Sometimes it may include changing localizable objects that are not text, such as data templates and patterns.

For example, some pseudo-localization tools will allow the user to specify a string of characters to be appended to the beginning of a text string and another to the end. When the tool is run against the text resources of the product, all text strings will have these characters appended to them. The intention is that the appended strings contain accented characters and characters from other scripts (such as Russian or Chinese).

Why it is used
The usual reason given for pseudo-localization testing is to determine whether the user interface text is localizable; that is, whether the text has been put into separate resource files to enable localization without changing the program code files. Since the translation is automated, this testing can be done much earlier in the development process than if the real localization were used.

Two other aspects are also commonly tested as an adjunct to the localizability area: whether strings have expansion room, and whether another character encoding is handled correctly.

But the truth is that there is much more pseudo-localization can help test. In this series, I'll explain how it can be used to test 9 areas of internationalization. Stay tuned.

Tribute to Unicode, or, Unic-Ode

(In honor of Unicode's 20th anniversary!)

Here's to Unicode, that internationalization panacea,
Let's hope that we'll never find need to replace ya!
Your uniformity, uniqueness, and most of all, universality
Make up your (ten) principled character personality.

On round trips you're compatible with the most obscure graphemes
Forcing them into normalcy, then encoding schemes.
With the patience of saints, it is no surprise
You have an RFC role to canonicalize.

You bring characters together in dynamic combinations
Assisting programs in correct interpretation
(But never insisting on specific presentations,
Instead leaving that to experts in localization.)

Shall I extol your streamlined Basic Multilingual Plane
Even as it goes beyond FFFF again and again?
About your capacity I can't be too complimentary
I mean, over a million possible characters supplementary!

All the world's modern and ancient scripts are included
(Though Klingon is one the committee eluded;
But still there's no need to resort to hysteria,
For one is equipped with a Private Use Area.)

Each character, too, has a property selection
Such as function and case, position and direction.
Bi-directionality I mean, the right-to-left, left-to-right one,
Not something as complex as boustrophedon.

Your versatility knows not a boundary;
Many character encodings have emerged from your foundry.
There are 8-, 16-, and 32-bit selections.
(The 7-bit was deprecated for everyone's protection.)
Though, due to certain platform vending and
Other factors, 16-bit is endian.

But Unicode, don't let this praise make you haughty,
For over the years you too have been naughty!
In case you've forgotten, the Korean move,
European currency symbol, and a few others prove
You're not without fault; but these foibles don't mask the
Incredible improvement over US-ASCII.

So here's to Unicode, and to Joe, Mark, and Lee,
For all their hard work and a very low fee.
This repertoire extraordinaire has made its place.
After all, where else could one find a zero-width no-break space?

(c)2008 I18n G.A.L. All rights reserved. The rest of her is less restrained.

Myth #14 - CEOs, something you should know (last of the old series)

This is, in fact, the final myth of the old series, and it's huge:

"Internationalization is only needed in the software development department."

I have sent Jonathan Schwartz (Sun Microsystems' CEO) exactly one email. I have no idea if he read it, but in it I wrote:

1. Internationalization is not a solved problem.
2. Internationalization cannot be accomplished by engineering alone.
3. Internationalization must be an integral part of everyone's job; it cannot be accomplished by separate people labeled "internationalization specialists".

Today's myth is really #2 in the email, but the other 2 items are related. What do I mean by saying that "internationalization cannot be accomplished by engineering alone"? After all, doesn't software internationalization mean development work? Well, yes it does, but not in a vacuum.

Imagine designing, architecting, and implementing a software product with no specification, no customer requirements, no input from marketing, no data from market research companies - would you do that? Doesn't make sense, does it? And yet that's exactly how we internationalization folks work. There is no-one in marketing to gather international market or customer data; very little international market research is available for the areas that internationalization needs to know; and we don't do much in the way of gathering input from our existing international customers. And yet amazingly, people ask me all the time what languages, writing systems, and locales we should support. Sure, I can use my judgement, just as any other development architect could make her best guess on what to do. But that's not a very good way to do business.

Software internationalization engineers are expected to not only know their specialty, but to know everything about how it is applied everywhere. Just yesterday I had an email exchange with a manager in user interface engineering (UIE). We have been talking about putting together training to help the UIE folks better understand internationalization issues in the user interface. But the word is that their director is not interested. Huh? Sooooo, what does that mean, that UIE designs interfaces that only serve 40% of our market, and that's OK? Maybe this director is a firm believer in Myth #13. I'm not a UIE expert and have no plans to become one. Nor do I plan to move into int'l marketing, nor int'l financial reporting, nor int'l product distribution, nor manufacturing, nor software release, nor any other aspect of getting an international software product out in the marketplace. But all of these areas need to understand how international affects what they do. In other words, everyone needs to internationalize.

Let me give you another example. The globalization team is always trying to report numbers illustrating the value we add. We would like to show sales figures based on the localizations we have completed. The problem is that we have no way of knowing what customers are using which language versions of our products. This is not something that can be discerned from part numbers, because we include many localizations with each product. What needs to happen is that we need to get feedback from customers on what language they use as the primary one for the product they purchased. Globalization is an engineering organization; we are not positioned to garner this kind of information. Other groups could do it by simply internationalizing what they already do.

Or we could just keep guessing.

Myth #13 - Software engineering managers and triskaidekaphobics, take care!

I am nearing the end of the myth series, but I won't end on 13!

"Internationalization is implemented after the base product and is written by a separate group of engineers."

This one really hurts. And it does take a lot of explaining, but I'll do what I can to condense it. An internationalized product is one that processes data from all over the world. Whatever the product is supposed to do with data, for example, receive and send emails, it can do it with data from lots of different places in lots of different languages using lots of different locale formats. In order for software to be able to do that, every place in the code where the data is touched needs to be internationalized. If the internationalization takes place after the "base" code is written, then that means that a developer is going over the code twice (at least). From a logical standpoint, would you pay development engineers to write the same code twice? That's a very expensive way to develop software. Moreover, would you let engineers who don't have a very thorough knowledge of your code work on the core functionality of your product? That would likely introduce a number of new bugs, some of them quite serious. Yet if you are writing a "base" product and then paying a 3rd party internationalization group to internationalize your code, that's exactly what's happening. And if that internationalized code isn't incorporated into the source tree for the next revision, then you're paying even more to have the same code reworked each time you put out a new version.

'But,' you may wonder, 'my development engineers aren't internationalization experts. How can they write internationalized code?' The answer is training. Internationalization is an aspect of good coding practice. That it isn't yet taught in universities is a real problem, and those of us in the industry are keenly aware of the issue. So you may need to make up for it in house. Set up internationalization training for all development engineers, test engineers, and engineering managers. There are a couple of vendors who provide these services (I don't have firsthand experience with any particular vendor). Furthermore, there is a tremendous amount of documentation in books and online, talking about the concepts as well as the implementation details involved in internationalization. Take a look at the Sun Globalization Resources, and the Java Internationalization site to start with. There are a couple of links on my blog page that have good information on internationalization. Have a look at my article, Internationalization in Software Design, Architecture, and Implementation, and take advantage of some of the mailing lists. In the end, it's much cheaper to train your own engineers than to pay for some other company to rewrite your code. And your engineers may be a little bit more content in their jobs when they're learning new things and have more control over their own code.

Try it out, and let me know how it works.

Myth #12 - More quality by the dozen

Continuing in the myth series, we address the scope of internationalization:

"My product works in Japanese, therefore it's internationalized."

Seems sort of logical doesn't it? I mean, if Japanese data gets processed and displayed correctly, surely that means the internationalization is correct? Alas, no. In fact, for a long time there was (and to some extent still is) a specialty called "Japanization". So many companies in the U.S. wanted to take advantage of the lucrative Japanese market that firms sprang up, offering this service. They would take the software and alter it to enable Japanese data processing and display on Japanese-capable machines. Sounds pretty good, huh? Except that it was expensive, time-consuming, and had to be repeated for each release. But we'll get into that myth later.

But let's say you didn't send it to a third party "Japanizer", and you really tried to do proper internationalization over the course of design and development. If you test using Japanese, then that must indicate everything is OK, right? Well, no. It's true that testing with Japanese can uncover some real internationalization problems, but not all of them. Arabic, Hindi, and French have different i18n issues from Japanese - even Chinese is different. What are these differences? These other languages often use different charsets, and in the case of Arabic and Hindi, different rendering capabilities. Beyond language, the Japan locale has its own data formats, which are not the same as France, or Brazil, or Russia, etc. And as a further area to test, Japan is within a single time zone, so you haven't tested a multiple time zone situation (a genuine problem for the U.S. with its 7 time zones).

But wait, there's more! Most customers require multilingual capabilities in their software, and that needs to be tested. For example, if you're working on server software, consider that companies will run the software on a server which is set to one language and locale (with the admin possibly using a different language for the console), but clients will be using lots of different languages and locales as they contact the server. This is a very common configuration nowadays. And it's not limited to servers. Just because someone is running the Italian localization of your email client doesn't mean that all their emails will be in Italian, or even Latin script. They may be emailing in Thai, and you need to verify that your product can handle these sorts of multilingual requirements.

For more information on testing the internationalization of software, see my article, Internationalizing Testing. And keep reading the myths!

Myth #11 - Ahhhh, elevenses, where everyone is literate in English™

Now we enter the land of fantasy, where eating pizza and ice cream causes weight loss and lowers blood cholesterol, and everyone is able to read and write English. Here we have a new myth in the series:

"If something is wrong, our customers will tell us."

The question is, what venues do you give them to tell you? And how do you know that the reason they didn't buy your product is because of certain internationalization (or other) problems? those are the two main stumbling blocks to overcome, but there are undertones as well.
Starting with the venues, have you provided your customer with easy ways to tell you about problems? That is:

• Do they know what URL to go to, what email address to mail to, and what number to call?
• Is the text at the URL translated into their language?
• Does the form at the URL accept text in their language?
• Is the person who answers the phone the person who helps them throughout the problem reporting process (regardless of who has the correct solution for the problem)?
• Does the person on the phone speak their language?
• Are emails in their language?
• Is there a discussion forum in their language, and is someone from your company monitoring it?
• Can your bug tracking system handle data in other languages (which your software processes)?
• Do you conduct seminars and conferences on your products in different parts of the world, and are there interpreters and translations of slides and papers into the local language(s)?

I'm sure there are loads more things that can be done, but this is what I mean by venue. Some of the above points are more general; making sure your customers (or potential customers) know exactly where to go to ask questions and report problems (they should be the same place, one contact point to your company, regardless of the communication) is basic customer service. Providing them different channels (Web, telephone, email) gives them the flexibility to choose what they are most comfortable with. Providing these venues in the customers' languages is also essential; it shows you care about them and you will listen to them. Even more fundamentally it enables them to communicate! Not all customers are comfortable enough with English to report a problem. And there are cultural issues to consider: many people don't want to embarrass themselves by using their so-so English, potentially creating a misunderstanding. In some cultures, pointing out a problem is an incredibly rude thing to do, even if it's a product they've paid good money for. They may be more inclined to quietly return the product or eat the cost and move to a competitor's product. In those cases you will never learn of the problem which lost you that customer, and maybe others, too.
This brings me to the second point, do you ever find out exactly why a potential customer never became an actual customer? Does your company do follow-up? Have you listened carefully to your potential customer's requirements - in their language, using their cultural conventions - and verified that your product fulfills them? Are you approaching potential customers in a culturally appropriate way? Are you advertising in the right fora (forums)?
It may sound like a lot of work, but the payback is huge. If you think your company does OK in this area, take a closer look. Talk to the people who live and work in other countries and get the real story. Most recently I heard from someone who worked in the support area in China. He knew of several cases where customers reported problems with the product running in the Chinese environment. Because the people trying to reproduce the problem didn't try it in a Chinese environment, they couldn't reproduce the problem, and closed the bug report! (Yes, this happened in 2004.)
Do you think those customers ever reported another bug? They may not even be customers anymore. This actually points to a future myth, that is, that internationalization is only necessary for software development. Until then.

Myth #10 - For Senior VPs, the power of 10

What I wouldn't give for VPs to read this blog! Actually, it'd be great if they read all the myths, but you can't have everything (where would you put it?)

"We added internationalization in the last release, so we're done."

I find it almost unfathomable that any exec would believe this, and yet I have heard it from more than one. Saying this is like saying "We wrote the code in the last release, so we're done." I hope that there isn't a software VP out there who would say that. Internationalization is inherent in the architecture, design, and implementation of a product. In reality it is part of the entire process of creating, distributing, and selling a product, but I'll just stick to the software development portion. When designing a product, international requirements must be considered in the design and architecture, otherwise you may have to redesign and rewrite the product to enable support of other language and locale data. I have seen it happen, and most recently, a product was scrapped because it wasn't worth rewriting it. What sort of things can trip you on design? Take a look at my article on this very topic:
Internationalization in Software Design, Architecture and Implementation

Every time there is a change in design, the addition of functionality, new code written, there is internationalization. It, too, like all aspects of a changing product, needs to change. It is part and parcel of the entire software development process, from requirements gathering, to design specification, to implementation, to testing, to release. Unless the product itself is done, internationalization work needs to continue.

And the myths go on (they go to 11) ...

Myth #9 - Number 9, number 9, number 9, number 9

(Play this blog backwards for the hidden messages.) As for the more overt message, I continue with the myths series:

"We've never localized this product/module/component/blidget, so it doesn't need internationalization."

It's a good thing that nothing ever changes in products, nor in markets. Oh, they do? Yes, even though your product has never been localized before, and this may be a real shock, it might be localized in the future. Whoa! And, another shock, localization is a business decision, not a technical decision. In other words, if a customer says, "We'll buy 2 million licenses for your product if you localize it into Cloqrat," you don't want to have to say, "Gee, sorry, it'll take a massive reworking of the code, say, 12 months to get that to you" since your customer will respond "That's OK, we'll just buy Microbrain's version" and then your problem will be solved because you won't ever see that customer again.

But there's more to it than that. Internationalization is a lot more than making the product localizable. It's primarily about data processing (see Myth #1). That is, even folks in the USA have to process data that is not US data. Was that another shock? I'm sorry; try some rooibos tea.

Oh no! Not Myth #8! Anything but Myth #8! ...Anything? ...OK, Myth #8

And now, we examine our heads, no, navels, no no, myths, we examine our myths (OK, can you tell I'm getting a little punchy here?):

"Administration interfaces don't need internationalization."

'Cause all sys admins everywhere speak, read, and write English fluently, don't they? You know, the funniest thing about this myth is that it's so often repeated, but I have yet to find any data, study, customer interview, or even efforts to obtain such, to support this myth. Maybe it's a mantra. In any case, the hard facts are that many admins are not that comfortable with English, or in some cases they don't know any at all. If you're charged with keeping a company's systems up and running, how keen are you to do that in an interface that is a second language? I thought so. Nothing like a message popping up on the screen with "Floozid iyarkaba panic gotrios piwec shutdown worqas!!" and there you are, madly flipping through your Cloqrat => English dictionary, trying to remember the conjugation of the verb gotrasco. And then more messages come flying across the screen...

The point being that admins are humans, just like you, and language has meaning for them too. They're going to function a lot better in a native language than in a second language, just like you. Many companies translate their admin interfaces - check out what your company does. And this actually leads into another myth, which I'll post at a later date, namely "We've never localized this before."

}sigh{

Myth #7 - IT depts., are you feeling lucky?

In this, the next installment (or instalment, if you prefer) of the
myths series, we explore something nearer to our hearts^H^H^H^H^H^Hselves:

"All company employees speak English, so only English needs to be supported by internal tools."

Seems pretty straightforward, huh? But the question is, what are internal tools used for? Even if all employees speak and read English, their data may not be in English. If they live in Beijing, it might make sense to store their Chinese addresses, for example.


Much of the data that internal tools process is not employee data, however. Many companies have bug tracking and reporting tools. These tools are used to log and report on bugs found in company products. Not only can those bugs originate from people outside the company who may not be able to write in English, but the problems can be regarding the processing of data other than English. If the products are internationalized, customers use them to process data from all over the world. If they encounter a problem, they're going to want to put in the data they were using when the problem occurred (if you're lucky). Without that data, you may not be able to reproduce the problem. Worse, if you don't allow that data, the customer may not bother to tell you about the bug at all!


Think about your internal customer databases - can they handle customer data in the local languages? This can be extremely important for customer relations and communications. What about Web based feedback? Could a Greek customer who doesn't write English send you some feedback? Wouldn't you want to know? Getting the feedback translated is a simple matter; getting the customer back after they left in frustration from not being able to communicate with your company is a lot harder and more expensive.


There are many other examples of internal tools that need internationalization: survey tools, forum discussion software, employee benefits tools (if you have worldwide employees), product registration tools, financial tracking tools (does all your revenue arrive in US dollars?), etc. The trick, and I know this is really tricky, is to carefully look at the tool requirements and all the data that the tool will be processing in the foreseeable future. Then design, architect, and implement to cover it. Piece of pie. Easy as cake.

Myth #6 - Web page authors, this one's fer you

For goodness' sake, how many myths are there? The answer is, as many as I have heard. But truly, there are fewer than 15*, so read on! This is one that keeps popping up like a bad penny:

"ISO-8859-1 is the standard encoding for HTML."

Sooooo, does that mean all those Web pages in Japanese and Chinese are a bunch of standard-violating hacks? No, of course not. It is perfectly legal to use any charset in a Web page, but it should be declared. Why? Because ISO-8859-1 is the default charset for HTML (yes, even in HTML 4.0). That means if you don't declare the charset of your page, a browser (or any other HTML interpreter) is free to assume that it's in ISO-8859-1. Now, admittedly, in practice browsers make other assumptions. Typically you set a preference for a default charset (or character encoding, if you prefer). This is sometimes set based on the localization you install; for example, if you install a Russian version of the browser, it may set the default charset as "KOI8-R". But the point is that assumptions will be made, unless you declare the charset in your document. And it's very straightforward. Just put a META tag as the first tag in the HEAD section, like so:

<META HTTP-EQUIV="Content-type" VALUE="text/html; charset=utf-8">

Simple, right? Oh yes, and I sneaked in a better charset to "default" to - UTF-8. UTF-8 is an encoding of Unicode, nearly universally supported, covering most of the living languages of the world. Use it and all your cares will be over - uh oh, see Myth #4.
* were fewer than 15

Myth #5 - For open source aficionados

Oh, the myths keep rolling along. This one is more and more commonly heard (used?) as open source is more and more commonly incorporated into company offerings.

"My product uses open source and so internationalization requirements don't apply."

Tell that to your customers when your software doesn't work for them.

A product is only as good as its weakest component. If you use open source in your product and it isn't internationalized, then it may be that your entire product can't handle international data. For example, say you base your product on one of the Linux flavors that isn't internationalized, and say your primary market is, oh, China (I hope a certain VP is reading this). Does that make sense? Why would Chinese customers run on a platform that doesn't support Chinese? Would you run on a platform that doesn't support English? My German is pretty near fluent, but I still run on an English platform. Even if I switch the interface to German, which I sometimes do, I make sure that it can process English correctly.

When producing a software product, all customer requirements should be considered. And whether you write your own code or pull it from open source, those requirements still count. If your market is worldwide, or even within the EU, internationalization is not a "nice-to-have", it's a must. So when choosing external components, be they from a vendor or from open source, consider the work it will take to get the internationalization up to snuff. Then make your decision.

Myth #4 - Are we having fun yet?

And so the myths go on, and internationalization folks keep chuckling with a look of irony on their faces. Here is Myth #4:

"My product supports Unicode and therefore it's internationalized."

Interesting. Of course, being a myth, it isn't true. Why isn't it true? Doesn't Unicode cover all languages worldwide? Would that it were that easy. In brief, Unicode is a coded character set (see RFC 2130 The Report of the IAB Character Set Workshop), that is, a set of characters associated with a sequence of integers. It defines some character encoding schemes (and forms, but we won't get into that) which take the values associated with the characters and translate them into bytecodes that computers can understand. OK, so what's all this got to do with the myth? The point is that only characters, or parts of characters, are encoded. There is no language information, no locale information, no font information, very little rendering information. What's more, the requirements for supporting Unicode are very lenient. In other words, your product can "support Unicode" and yet only recognize a single character. So, if a product only supports Unicode for its internationalization, not only are you missing information vital to internationalization, but you might not even handle the languages that Unicode covers.

And therein lies the myth. By all means, support Unicode. Support all the currently assigned characters. Unicode is a handy tool for supporting languages around the world. But it is not an internationalization silver bullet panacea elixir cure-all.