IMPORTANT NOTICE - Apr-30-2009:
This .NET implementation of Okapi is no longer actively developed.
Instead, a NEW JAVA IMPLEMENTATION IS AVAILABLE and is being actively developed.
The material on this Web site is for archive propose. Some applications of the old .NET implementation (e.g. Olifant) will be maintained to some degree until they have a replacement in the Java project.

Okapi Framework

Filter Item Interface Specification

Revision information: Version 1.1
     Latest revision: http://okapi.sourceforge.net/IFilterItem.html

- Document Status
- Overview
- Coded Text
- Defined Values
- Methods

Document Status

This document is a stable specification. Feedback about the content of this document is encouraged. Send your comments to the Okapi Framework administrator, or alternatively to post them in the Okapi Tools users group.

DISCLAIMER: Parts of the documents are generated automatically from the source code documentation of Okapi's implementation of the specification. Because the generation tool is not completely working yet, this may result in incomplete or broken text or links in this document. See the source code documentation of the interface for complete text.

Overview

The Okapi Filter Item Interface is an object model that provides a common way of accessing the different data gathered when processing an input for a localization task. A object implementing this interface is usually used along with an object that implements the Okapi Filter Interface.

The Filter Item is meant to be utilized by two different type of users: the developers of filters (i.e. the providers), and the developers of applications using the filters (i.e. the consumers). While all the methods of the Filter Item interface are accessible to both, some are designed more for or or the other class of users.

Other related specifications:

• Log Interface
• Localization Directives Interface
• Filter Interface
• Utility Set Interface

Coded Text

A text item is composed of two parts: the text itself and, in some cases, the inline codes within the text. These codes are the tags or codes of the original material that have been parsed and isolated from the rest of the text. For example, in the HTML string "Text in <b>bold</b>." there are two inline codes: "<b>" and "</b>". An important aspect of the Filter Item interface is that inline codes are abstracted and can be manipulated regardless of what they are in the original format.

A text item can be serialized into two strings: the first one is the coded text string that holds text and abstracted inline codes, and the second is the codes buffer which holds a string representation of the original inline codes and information about them. Having access to these two strings allows you, for example, to store the items in a database and reconstruct the item later on.

Use the GetText method (with its parameter set to zero) to obtain the coded text string, and use the GetCodeMapping method to obtain the string representation of all the inline codes. You can the SetText and SetCodeMapping method to reconstruct the item, as shown below:

string sText = myFilterItem.GetText(0);
string sCodes = myFilterItem.GetCodeMapping();

myFilterItem.SetText(sText);
myFilterItem.SetCodeMapping(sCodes);

Important: The SetText method resets the inline codes internal structure, so you must call those two methods in that order: SetText, then SetCodeMapping.

Most of the time you will not have to deal directly with the codes buffer, and manipulate only the coded text. To set the coded text back to an item after having made some modifications, you can use the ModifyText method, as long as you have not modified any of the inline codes inside that string (except possibly to move them around). For example:

string sText = myFilterItem.GetText(0);
sText = sText.Replace("black", "white");
myFilterItem.ModifyText(sText);

Coded Text Format

In the coded text representation each inline code is abstracted as a pair of special characters. The first one is the inline code type indicator, and the second the index of the code. Both special characters are encoded in the Unicode Private Use Area:

• The code type has a value of U+E101 (begin paired tag), U+E102 (end paired tag), or U+E103 (placeholder). they correspond to the <bpt>, <ept>, and <ph> elements in XLIFF.
• The code index is a zero-based index offset with the value U+E200. So the first code has the index U+E200, the second U+E201, the third U+202, and so forth. You can access more code data using methods such as GetCode, the index parameter must be zero-based.

For example the text "Text in <b>bold</b>." will be coded as follow:

The <b> tag is represented as a generic begin paired tag (U+E101) with an index of 0 (U+E200 once converted to the offset representation). You can obtain the coded text representation by making a GetText(0) call.

Using the Unicode Private Use Area for encoding the inline codes has the advantage of allowing most of the "normal" string functions. For example, you can run regular expressions on the coded text without risking to modify the inline codes while changing the text portions. The codes themselves can also be matched fairly easily by regular expressions too.

A function to convert the code index in the string representation to the zero-based index to use with GetCode would look like this:

int ToIndex (char p_chIndex)
{
   return ((int)p_chIndex)-0xE200;
}

Codes Mapping Format

The information corresponding to the inline codes of a Filter Item can be stored in a single string representation. You can retrieve this string with the GetCodeMapping method. You can also assign the string back to a Filter Item by using the SetCodeMapping method. Note that, when using this method, the inline markers in the coded text must correspond to the codes of the string.

Line-Breaks

All line-breaks in a coded text are always represented by a single line-feed character (\n, i.e. U+000A), regardless of the original type of line-break used in the input. It is the responsibility of the filter to perform the relevant conversion as needed.

Uasge Example

Example, a very basic word count function:

int WordCount (IFilterItem p_FI)
{
   bool                  bInWord = false;
   int                   nCount = 0;
   string                sText = p_FI.GetText(0);

   for ( int i=0; i<sText.Length; i++ )
   {
      switch ( sText[i] )
      {
         case '\xE101': // Begin paired tag
         case '\xE102': // End paired tag
         case '\xE103': // place-holder
            i++; // Skip code index
            // We ignore inline codes for the count
            continue;

         default:
            if ( Char.IsWhiteSpace(sText[i]) )
            {
               if ( bInWord )
               {
                  nCount++;
                  bInWord = false;
               }
            }
            else bInWord = true;
            continue;
      }
   }

   // End string ends a word?
   if ( bInWord ) nCount++;

   return nCount;
}

Defined Values

TODO

Methods

The Filter Item Interface provides the following methods:

IFilterItem.GetInterfaceVersion

Gets the version of the IFilterItem interface the object implements.

Return

The version of the IFilterItem interface.

IFilterItem.Reset

Resets most of the states and values of the item to their defaults.

Remarks

Use this method to reset most of the states and values associated with the filter item to their default values.

IFilterItem.SetRTFOptions

Sets the options for the RTF formatting.

Parameters

Remarks

Use this method to set the RTF codes to use before and after each inline code. These codes are used to format the text to RTF when using the IFilterItem.GetText method. The RTF sequences must start with a '}' that will be closed automatically at the end of the code.

IFilterItem.CopyFrom

Copies all the data from a specified filter item.

Parameters

Remarks

Use this method to duplicate all the content of an item.

Example

            if ( Flt.ReadNextItem() == FilterItemType.TEXT )
            {
               FilterItem FI = new FilterItem();
               FI.CopyFrom(Flt.GetItem());
               // Now FI and Flt.Getitem() are independent of each other
               // can be modified separately.
            }
            

IFilterItem.Extract

Extract a part of the text from the item.

Parameters

Return

The new filter item containing a copy of the original one, with only the part of text to extract.

Remarks

Use this method to split a filter item. The inline codes in both the original item and the returned item are adjusted as needed. If the p_nStart is out of bounds, or if p_nLength is invalid the method returns null.

IFilterItem.GetItemType

Gets the type of the filter item.

Return

The type of the filter item. The value is one of the Filter.FilterItemType values.

Remarks

Depending on the filter item type, different properties are accessible in the filter item.

IFilterItem.SetItemType

Sets the type of the filter item.

Parameters

Remarks

The default type of the filter item is the value 0 (ERROR).

IFilterItem.GetItemID

Gets the ID of the item.

Return

The ID of the item.

Remarks

This value is unique within the input. It can be used for the value of the id attribute in XLIFF.

IFilterItem.SetItemID

Sets the ID of the item.

Parameters

Remarks

IFilterItem.GetGroupID

Gets the ID of the group to which the item belongs.

Return

the ID value of the group to which the item belongs.

Remarks

Group IDs are unique within the input. If the item belongs to no group, the ID is 0.

IFilterItem.SetGroupID

Sets the ID of the group to which the item belongs.

Parameters

Remarks

Use this method to set the ID of the group to which the item belongs.

IFilterItem.GetLevel

Gets the level of grouping.

Return

The level of grouping. The value starts at 0 (no group).

Remarks

The level indicator is incremented for each embedded group. The start group and end group items themselves are one level above the items inside the group they delimit. The first level (items not inside any group) is level 0.

IFilterItem.SetLevel

Sets the level of grouping.

Parameters

Remarks

The level value is zero-based (no group).

IFilterItem.GetEncoding

Gets the name of the encoding used for the item.

Return

The encoding name. The value is one of the IANA charset names.

Remarks

Some output (like RTF) require to know in which encoding the text is output. The default value is the name of the default encoding for the current user.

IFilterItem.SetEncoding

Sets the name of the encoding used for the item.

Parameters

Remarks

Use this method to set the name of the encoding the item will use for some of its output (e.g. RTF).

IFilterItem.GetLineBreak

Gets the type of line-break for this item.

Return

The line-break representation.

Remarks

IFilterItem.SetLineBreak

Sets the type of line-break for this item.

Parameters

Remarks

Use this method to set the type of line-break to use with the item.

IFilterItem.GetXMLStyle

Gets the indicator for XML style for this item.

Return

True if the item is XML text, false otherwise.

Remarks

IFilterItem.SetXMLStyle

Sets the indicator whether the item uses XML syntax or not.

Parameters

Remarks

Use this method to set whether or the text will be treatted for XML escapes in output.

IFilterItem.GetResType

Gets the resource type of the item.

Return

The resource type of the item.

Remarks

This value is the equivalent to the value of the restype attribute in XLIFF.

IFilterItem.SetResType

Sets the resource type of the item.

Parameters

Remarks

Use this method to set the resource type of the item.

IFilterItem.GetResName

Gets the resource name of the item.

Return

The resource name of the item.

Remarks

This value is the equivalent to the value of the resname attribute in XLIFF.

IFilterItem.SetResName

Sets the resource name of the item.

Parameters

Remarks

Use this method to set the resource name of the item.

IFilterItem.GetMimeType

Gets the mime-type value of the item.

Return

The mime-type value of the item.

Remarks

This value is used for BINARY items.

IFilterItem.SetMimeType

Sets the mime-type value of the item.

Parameters

Remarks

Use this method to associate a mime-type value with a BINARY item.

IFilterItem.HasCoord

Indicates whether at least one of the coordinates member is different from 0.

Return

True if one or more of the coordinates member is different from 0, false if they are all set to 0.

IFilterItem.GetCoord

Gets the coordinates of the item.

Return

The string representation of the coordinates.

Remarks

This value is equivalent to the coord attribute in XLIFF. You can use the IFilterItem.HasCoord property to see whether the item has any member of the coordinates different from 0. You can access each coordinates member individually using the IFilterItem.GetX, IFilterItem.GetY, IFilterItem.GetCX, and IFilterItem.GetCY methods.

IFilterItem.SetCoord

Sets the coordinates of the item.

Parameters

Remarks

Use this method to set the coordinates of the item. You can also set each coordinate members individually using the IFilterItem.SetX, IFilterItem.SetY, IFilterItem.SetCX, and IFilterItem.SetCY methods.

IFilterItem.GetX

Gets the X coordinate of the item.

Return

The X coordinate of the item.

IFilterItem.SetX

Sets the X coordinate of the item.

Parameters

IFilterItem.GetY

Gets the Y coordinate of the item.

Return

The Y coordinate of the item.

IFilterItem.SetY

Sets the Y coordinate of the item.

Parameters

IFilterItem.GetCX

Gets the CX coordinate of the item.

Remarks

The CX coordinate of the item.

IFilterItem.SetCX

Sets the CX coordinate of the item.

Parameters

IFilterItem.GetCY

Gets the CY coordinate of the item.

Return

The CY coordinate of the item.

IFilterItem.SetCY

Sets the CY coodinate of the item.

Parameters

IFilterItem.HasFont

Indicates whether the item has font information.

Return

True if the item has font information, false if not.

IFilterItem.GetFont

Gets the font information of the item.

Return

The font information for the item.

Remarks

This value is equivalent to the font attribute in XLIFF. You can use the IFilterItem.HasFont property to query whether the item has font information.

IFilterItem.SetFont

Sets the font information for the item.

Parameters

Remarks

Use this method to set the font information for the item. The string representation of the font information is the same as for the font attribute in XLIFF.

IFilterItem.IsTranslatable

Indicates whether the text is translatable.

Return

True if the text of the item is translatable, false if not.

Remarks

Use this method to query whether the item is translatable or not.

IFilterItem.SetTranslatable

Sets the indicator whether the text of the item is translatable or not.

Parameters

Remarks

Use this method to flag the item as translatable or not.

IFilterItem.IsTranslated

Indicates whether the text of the item has already a translation.

Return

True if the text of the item has a translation, false if not.

Remarks

Some file formats have bilingual content and may have both a source text and its translation. Such items are normally considered having an existing translation if the part where the translation text should be is 1) not empty or 2) has a text different from the source text. Use the IFilter.GetTranslatedItem method of the filter to retrieve the translated text item.

IFilterItem.SetTranslated

Sets the indicator whether the text of the item has already a translation.

Parameters

Remarks

Use this method to set the status of the target text in bilingual files.

IFilterItem.IsSubFlow

Indicates whether the text of the item is a sub-flow text.

Return

True if the item is a sub-flow item, false if not.

IFilterItem.SetSubFlow

Sets the indicator whether the text of the item is a sub-flow text.

Parameters

IFilterItem.IsPreFormatted

Indicates whether the text is pre-formatted.

Return

True if the text is pre-formatted, false if not.

IFilterItem.SetPreFormatted

Sets the indicator whether the text is pre-formatted.

Parameters

IFilterItem.GetStart

Gets the start position of the text of the item in the input.

Return

The start position of the text of the item in the input.

Remarks

The position returned is an offset from the beginning of the input. The value is -1 if the start position is not known.

IFilterItem.SetStart

Sets the start position of the text of the item in the input.

Parameters

Remarks

Use this method to set the start position of the text of the item (the position of the first character) in the input. The value must be set to -1 if the start position is not known.

IFilterItem.GetLength

Gets the original length of the text of the item in the input.

Return

The original length of the text of the item in the input.

Remarks

The value returned by this method is not always the same as the value returned by a call to IFilterItem.GetTextLength for the original format. The parsed text may have been transformed, for example to un-wrap a paragraph. The value returned by this method is the length of the text as it is in the input.

IFilterItem.SetLength

Sets the length of the text of the item in the input.

Parameters

IFilterItem.IsEmpty

Indicates whether the item is empty.

Return

True if the item has no text and no inline codes, false if it has text or inline code(s).

IFilterItem.GetText

Gets the text of the item in a specified format.

Parameters

Return

The text of the item in the specified format.

IFilterItem.GetTextLength

Gets the character length of the text of the item for a specified format.

Parameters

Return

The length of the text of the item in the specified format.

IFilterItem.SetText

Sets the text of the item.

Parameters

Remarks

This method sets the text for the item, and reset any inline codes. The p_sValue paramter must not contain inline codes. To set inline codes to an item use the IFilterItem.AppendCode method.

IFilterItem.ModifyText

Modify the text of the item.

Parameters

IFilterItem.AppendText

Appends a string to the text of the item.

Parameters

Remarks

Use this method to add text at the end of the current text. To add an inline code use the IFilterItem.AppendCode method. To add a character use the IFilterItem.AppendChar method.

IFilterItem.AppendChar

Appends a character to the text of the item.

Parameters

Remarks

Use this method to add a character at the end of the current text. To add an inline code use the IFilterItem.AppendCode method. To add a run of text use the IFilterItem.AppendText method.

IFilterItem.AppendCode

Appends an inline code to the text of the item.

Parameters

Remarks

Use this method to add an inline code at the end of the current text. To add a character use the IFilterItem.AppendChar method. To add a run of text use the IFilterItem.AppendText method.

IFilterItem.ChangeToCode

Change a text section of a a text item to inline code.

Parameters

Return

The length difference of the coded text between before and after the insertion.

Remarks

Use this method to insert a code within a text item.

IFilterItem.HasCode

Indicator whether the item has at least one inline code.

Return

True if the item has at least one inline code.

Remarks

Use this method to query whether the text item contains at least one inline code. You can obtain the number of inline codes in the item with the IFilterItem.GetCodeCount method.

IFilterItem.GetCodeCount

Gets the number of inline codes in the text of the item.

Return

The number of inline codes in the text of the item.

Remarks

Use this method to get the number of inline codes in the text item. You can also query whether there is at least one inline code by using the IFilterItem.HasCode method.

IFilterItem.GetCodeMapping

Gets the string representation of the inline codes for the item.

Return

The string representation of the inline codes for the item.

Remarks

You can use this string for storing the inline codes and the coded text string in separate location (e.g. in a database).

IFilterItem.SetCodeMapping

Sets the string representation of the inline codes for the item.

Parameters

Remarks

Use this method to set the string representation of all the inline codes in the text of the item.

IFilterItem.GetCode

Gets the original code of a specified inline code.

Parameters

Return

String that contains the code for the specified inline code.

Remarks

Use this method to retrieve the data associated with a given inline code. The value returned is the one that was set when calling the IFilterItem.AppendCode method.

IFilterItem.GetCodeID

Gets the identifier for the code at the specified index.

Parameters

Return

The identifier of the specified inline code.

Remarks

Use this method to retrieve the identifier of a given inline code.

IFilterItem.GetCodeIndexFromID

Gets the index of a code with the given identifier and type.

Parameters

Return

Index of the code in the code map, or -1 if no match was found.

Remarks

Use this method to retrieve the mapping index of a given code. Be aware that the result of this function may be incorrect if you use it on an item that is currently being built because the index values may change when adding codes to the item.

IFilterItem.GetCodeLabel

Gets the label for the code at the specified index.

Parameters

Return

The label of the specified inline code.

Remarks

Use this method to retrieve the label of a given inline code.

IFilterItem.HasText

Indicator whether the item has at least one text character.

Parameters

Return

True if the item has at least one character (including spaces).

Remarks

Use this method to query whether the text item contains text.

IFilterItem.RemoveEnd

Removes a specified count of characters form the end of the text.

Parameters

Remarks

This method removed p_nCount characters or less from the end of the text. It stops when the count is reached, or when an inline code is reached, or when the start of the text is reached, whichever comes first.

IFilterItem.HasNote

Indicates whether the item has a note.

Return

True if the item has a note, false if not.

IFilterItem.GetNote

Gets the note information for the item.

Return

The text of the note.

IFilterItem.GetNoteAttributeFrom

Gets the from attribute of the note.

Return

The value of the from attribute.

Remarks

Use this method to get the from attribute value of the note. This attribute is similar to the from attribute in XLIFF.

IFilterItem.SetNote

Sets the nore information for the item.

Parameters

IFilterItem.ListProperties

Lists the current properties of the filter item.

Return

A string containing the name of each property separated by a semi-colon (e.g. "date;time;user"). If there are no properties, this method should return and empty string (not null).

IFilterItem.GetProperty

Gets the property value for the given property name.

Parameters

Return

Value of the property, or null if the property is not set.

Remarks

Use this method to retrieve a non-standard property associated to the extracted item. The name of the property is case-sensitive.

IFilterItem.SetProperty

Sets the property value for a given property name.

Parameters

Remarks

This method, along with IFilterItem.GetProperty and IFilterItem.ListProperties, allows manage non-standard properties with the extracted item.

IFilterItem.NormalizeLineBreaks

Normalizes the type of line breaks in the current item.

Remarks

This method convert Unix, Windows, and Mac linebreaks into a simple LF character.

IFilterItem.NormalizeWhiteSpaces

Normalizes the whitespaces in the current item.

Remarks

This method reduces sequencial whitespaces into a single space.