Saxonica.com

Implementing a collating sequence

Collations used for comparing strings can be specified by means of a URI. A collation URI may be used as an argument to many of the standard functions, and also as an attribute of xsl:sort in XSLT, and in the order by clause of a FLWOR expression in XQuery.

Saxon provides a range of mechanisms for binding collation URIs. The language specifications simply say that collations used in sorting and in string-comparison functions are identified by a URI, and leaves it up to the implementation how these URIs are defined.

There is one predefined collation that cannot be changed. This is the Unicode Codepoint Collation defined in the W3C specifications http://www.w3.org/2005/xpath-functions/collation/codepoint. This collates strings based on the integer values assigned by Unicode to each character, for example "ah!" sorts before "ah?" because the Unicode codepoints for "ah!" are (97, 104, 33) while the codepoints for "ah?" are (97, 104, 63).

In addition, by default, Saxon allows a collation URI to take the form http://saxon.sf.net/collation?keyword=value;keyword=value;.... The query parameters in the URI can be separated either by ampersands or semicolons, but semicolons are usually more convenient.

The same keywords are available on the Java and .NET platforms, but because of differences in collation support between the two platforms, they may interact in slightly different ways. The same collation URI may produce different sort orders on the two platforms. (One noteworthy difference is that the Java collations treat spaces as significant, the .NET collations do not.)

The keywords available are as follows:

keyword

values

effect

class

fully-qualified Java class name of a class that implements java.util.Comparator.

This parameter should not be combined with any other parameter. An instance of the requested class is created, and is used to perform the comparisons. Note that if the collation is to be used in functions such as contains() and starts-with(), this class must also be a java.text.RuleBasedCollator. This approach allows a user-defined collation to be implemented in Java.

This option is also available on the .NET platform, but the class must implement the Java interface java.util.Comparator.

rules

details of the ordering required, using the syntax of the Java RuleBasedCollator

This defines exactly how individual characters are collated. (It's not very convenient to specify this as part of a URI, but the option is provided for completeness.)

This option is also available on the .NET platform, and if used will select a collation provided using the GNU Classpath implementation of RuleBasedCollator. At the time of writing, this is not 100% compatible with the Sun JDK implementation.

lang

any value allowed for xml:lang, for example en-US for US English

This is used to find the collation appropriate to a Java locale or .NET culture. The collation may be further tailored using the parameters described below.

ignore-case

yes, no

Indicates whether the upper and lower case letters are considered equivalent. Note that even when ignore-case is set to "no", case is less significant than the actual letter value, so that "XPath" and "Xpath" will appear next to each other in the sorted sequence.

On the Java platform, setting ignore-case sets the collation strength to secondary.

ignore-modifiers

yes, no

Indicates whether non-spacing combining characters (such as accents and diacritical marks) are considered significant. Note that even when ignore-modifiers is set to "no", modifiers are less significant than the actual letter value, so that "Hofen" and "Höfen" will appear next to each other in the sorted sequence.

On the Java platform, setting ignore-case sets the collation strength to primary.

ignore-symbols

yes, no

Indicates whether symbols such as whitespace characters and punctuation marks are to be ignored. This option currently has no effect on the Java platform, where such symbols are in most cases ignored by default.

ignore-width

yes, no

Indicates whether characters that differ only in width should be considered equivalent.

On the Java platform, setting ignore-width sets the collation strength to tertiary.

strength

primary, secondary, tertiary, or identical

Indicates the differences that are considered significant when comparing two strings. A/B is a primary difference; A/a is a secondary difference; a/ä is a tertiary difference (though this varies by language). So if strength=primary then A=a is true; with strength=secondary then A=a is false but a=ä is true; with strength=tertiary then a=ä is false.

This option should not be combined with the ignore-XXX options. The setting "primary" is equivalent to ignoring case, modifiers, and width; "secondary" is equivalent to ignoring case and width; "tertiary" ignores width only; and "identical" ignores nothing.

decomposition

none, standard, full

Indicates how the collator handles Unicode composed characters. See the JDK documentation for details. This option is ignored on the .NET platform.

alphanumeric

yes, no

If set to yes, the string is split into a sequence of alphabetic and numeric parts (a numeric part is any consecutive sequence of ASCII digits; anything else is considered alphabetic). Each numeric part is considered to be preceded by an alphabetic part even if it is zero-length. The parts are then compared pairwise: alphabetic parts using the collation implied by the other query parameters, numeric parts using their numeric value. The result is that, for example, AD985 collates before AD1066.

Note that an alphanumeric collation cannot be used in conjunction with functions such as contains() and substring-before().

case-order

upper-first, lower-first

Indicates whether upper case letters collate before or after lower case letters.

This format of URI, http://saxon.sf.net/collation?keyword=value;keyword=value;..., is handled by Saxon's default CollationURIResolver. It is possible to replace or supplement this mechanism by registering a user-written CollationURIResolver. This must be an implementation of the Java interface net.sf.saxon.sort.CollationURIResolver, which only requires a single method, resolve(), to be implemented. The result of the method is in general a Java Comparator, though if the collation is to be used in functions such as contains() which match parts of a string rather than the whole string, then the result must also be an instance of either java.text.RuleBasedCollator, or of the Saxon interface net.sf.saxon.sort.SubstringMatcher.

In the Java API, a user-written CollationURIResolver is registered with the Configuration object, either directly or in the case of XSLT by using the JAXP setAttribute() method on the TransformerFactory (the relevant property name is FeatureKeys.COLLATION_URI_RESOLVER). This applies to all stylesheets and queries compiled and executed under that configuration.

In addition, the APIs provided for executing XPath and XQuery expressions allow named collations to be registered by the calling application, as part of the static context.

For XQuery, the class StaticQueryContext also allows collations to be registered directly by individual names. The system attempts to resolve a URI using these directly-registered names before it invokes the CollationURIResolver.

At present there are no equivalent facilities in the .NET API, though it is possible to manipulate collations by "dropping down" into the Java interface.

In Saxon XSLT stylesheets, collations may also be described using a saxon:collation element as a top-level declaration in the stylesheet. In this case the value of the name attribute of the saxon:collation may be used as a collation URI. There is no constraint on the form this URI takes, but it must be a legal URI. If it is a relative URI, it is interpreted as being relative to the base URI of the stylesheet. See saxon:collation for more details.

Next