The Input Method Protocol Version 1.0 X Consortium Standard X Version 11, Release 6.4

Masahiko Narita
FUJITSU Limited.
Hideki Hiura
SunSoft, Inc.

ABSTRACT

This specifies a protocol between IM library and IM (Input Method) Server for internationalized text input, which is independent from any specific language, any specific input method and the transport layer used in communication between the IM library and the IM Server, and uses a client-server model. This protocol allows user to use his/her favorite input method for all applications within the stand-alone distributed environment.

X Window System is a trademark of X Consortium, Inc.

Copyright © 1993, 1994 by X Consortium, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Except as contained in this notice, the name of the X Consortium shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization from the X Consortium.

Copyright © 1993, 1994 by FUJITSU LIMITED

Copyright © 1993, 1994 by Sun Microsystems, Inc.

Permission to use, copy, modify, and distribute this documentation for any purpose and without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. Fujitsu and Sun Microsystems make no representations about the suitability for any purpose of the information in this document. This documentation is provided as is without express or implied warranty.

1. Introduction1.1. ScopeThe internationalization in the X Window System Version 11,Release 5 (X11R5) provides a common API which applicationdevelopers can use to create portable internationalizedprograms and to adapt them to the requirements of differentnative languages, local customs, and character stringencodings (this is called ‘‘localization’’). As one of itsinternationalization mechanisms X11R5 has defined afunctional interface for internationalized text input,called XIM (X Input Method).When a client-server model is used with an IM (Input Method)implementation, a protocol must be established between theclient and the server. However, the protocol used tointerface Input Method Servers (IM Servers) with the InputMethod libraries (IM libraries) to which applications arelinked was not addressed in X11R5. This led applicationdevelopers to depend on vendor-specific input methods,decreased the user’s choice of available input methods, andmade it more difficult for developers to create portableapplications. This paper describes the Input Method Protocoldeveloped for X11R6 to resolve the above problems and toaddress the requirements of existing and future inputmethods.The Input Method Protocol is independent from the transportlayer used in communication between the IM library and theIM Server. Thus, the input method protocol can be built onany inter-process communication mechanism, such as TCP/IP orthe X protocol.In addition, the protocol provides for future extensionssuch as differing input model types.1.2. BackgroundText input is much more simple for some languages thanothers. English, for instance, uses an alphabet of amanageable size, and input consists of pressing thecorresponding key on a keyboard, perhaps in combination witha shift key for capital letters or special characters.Some languages have larger alphabets, or modifiers such asaccents, which require the addition of special keycombinations in order to enter text. These input methodsmay require ‘‘dead-keys’’ or ‘‘compose-keys’’ which, whenfollowed by different combinations of key strokes, generatedifferent characters.Text input for ideographic languages is much less simple.In these languages, characters represent actual objectsrather than phonetic sounds used in pronouncing a word, andthe number of characters in these languages may continue togrow. In Japanese, for instance, most text input methodsinvolve entering characters in a phonetic alphabet, afterwhich the input method searches a dictionary for possibleideographic equivalents (of which there may be many). Theinput method then presents the candidate characters for theuser to choose from.In Japanese, either Kana (phonetic symbols) or Roman lettersare typed and then a region is selected for conversion toKanji. Several Kanji characters may have the same phoneticrepresentation. If that is the case with the string entered,a menu of characters is presented and the user must choosethe appropriate one. If no choice is necessary or apreference has been established, the input method does thesubstitution directly.These complicated input methods must present stateinformation (Status Area), text entry and edit space(Preedit Area), and menu/choice presentations (AuxiliaryArea). Much of the protocol between the IM library and theIM Server involves managing these IM areas. Because of thesize and complexity of these input methods, and because ofhow widely they vary from one language or locale to another,they are usually implemented as separate processes which canserve many client processes on the same computer or network.1.3. Input Method StylesX11 internationalization support includes the following fourtypes of input method:- on-the-spot: The client application is directedby the IM Server to display allpre-edit data at the site of textinsertion. The client registerscallbacks invoked by the inputmethod during pre-editing.- off-the-spot: The client application providesdisplay windows for the pre-editdata to the input method whichdisplays into them directly.- over-the-spot: The input method displays pre-editdata in a window which it brings updirectly over the text insertionposition.- root-window: The input method displays allpre-edit data in a separate area ofthe screen in a window specific tothe input method.Client applications must choose from the available inputmethods supported by the IM Server and provide the displayareas and callbacks required by the input method.2. Architecture2.1. Implementation ModelWithin the X Window System environment, the following twotypical architectural models can be used as an inputmethod’s implementation model.- Client/Server model:A separate process, the IM Server,processes input and handlespreediting, converting, andcommitting. The IM library withinthe application, acting as clientto the IM Server, simply receivesthe committed string from the IMServer.- Library model: All input is handled by the IMlibrary within the application.The event process is closed withinthe IM library and a separate IMServer process may not be required.Most languages which need complex preediting, such as Asianlanguages, are implemented using the Client/Server IM model.Other languages which need only dead key or compose keyprocessing, such as European languages, are implementedusing the Library model.In this paper, we discuss mainly the Client/Server IM modeland the protocol used in communication between the IMlibrary (client) and the IM Server.2.2. Structure of IMWhen the client connects or disconnects to the IM Server, anopen or close operation occurs between the client and the IMServer.The IM can be specified at the time of XOpenIM() by settingthe locale of the client and a locale modifier. Since the IMremembers the locale at the time of creation XOpenIM() canbe called multiple times (with the setting for the localeand the locale modifier changed) to support multiplelanguages.In addition, the supported IM type can be obtained usingXGetIMValues().The client usually holds multiple input (text) fields. Xlibprovides a value type called the ‘‘Input Context’’ (IC) tomanage each individual input field. An IC can be created byspecifying XIM using XCreateIC(), and it can be destroyedusing XDestroyIC().The IC can specify the type of IM which is supported by XIMfor each input field, so each input field can handle adifferent type of IM.Most importantly information such as the committed stringsent from the IM Server to the client, is exchanged based oneach IC.Since each IC corresponds to an input field, the focusedinput field should be announced to the IM Server usingXSetICFocus(). (XUnsetICFocus() can also be used to changethe focus.)2.3. Event Handling ModelExisting input methods support either the FrontEnd method,the BackEnd method, or both. This protocol specificallysupports the BackEnd method as the default method, but alsosupports the FrontEnd method as an optional IM Serverextension.The difference between the FrontEnd and BackEnd methods isin how events are delivered to the IM Server. (Fig. 1)2.3.1. BackEnd MethodIn the BackEnd method, client window input events are alwaysdelivered to the IM library, which then passes them to theIM Server. Events are handled serially in the orderdelivered, and therefore there is no synchronization problembetween the IM library and the IM Server.Using this method, the IM library forwards all KeyPress andKeyRelease events to the IM Server (as required by the EventFlow Control model described in section 2.4. ‘‘Event FlowControl’’), and synchronizes with the IM Server (asdescribed in section 4.16. ‘‘Filtering Events’’).2.3.2. FrontEnd MethodIn the FrontEnd method, client window input events aredelivered by the X server directly to both the IM Server andthe IM library. Therefore this method provides much betterinteractive performance while preediting (particularly incases such as when the IM Server is running locally on theuser’s workstation and the client application is running onanother workstation over a relatively slow network).However, the FrontEnd model may have synchronizationproblems between the key events handled in the IM Server andother events handled in the client, and these problems couldpossibly cause the loss or duplication of key events. Forthis reason, the BackEnd method is the core methodsupported, and the FrontEnd method is made available as anextension for performance purposes. (Refer to Appendix A formore information.) 1

X Input Method Protocol X11, Release 6.1

... 0.05 6.513 4.737 10.45 ... 0.000i 3.937i 4.687i 0.000i

Fig.1 The Flow of Events

2.4. Event Flow ControlThis protocol supports two event flow models forcommunication between the IM library and the IM Server(Static and Dynamic).Static Event Flow requires that input events always be sentto the IM Server from the client.Dynamic Event Flow, however, requires only that those inputevents which need to be processed (converted) be sent to theIM Server from the client.For instance, in the case of inputing a combination of ASCIIcharacters and Chinese characters, ASCII characters do notneed to be processed in the IM Server, so their key eventsdo not have to be sent to the IM Server. On the other hand,key events necessary for composing Chinese characters mustbe sent to the IM Server.Thus, by adopting the Dynamic Event Flow, the number ofrequests among the X Server, the client, and the IM Serveris significantly reduced, and the number of context switchesis also reduced, resulting in improved performance. The IMServer can send XIM_REGISTER_TRIGGERKEYS message in order toswitch the event flow in the Dynamic Event Flow.The protocol for this process is described in section 4.5.‘‘Event Flow Control’’.3. Default Preconnection ConventionIM Servers are strongly encouraged to register theirsymbolic names as the ATOM names into the IM Serverdirectory property, XIM_SERVERS, on the root window of thescreen_number 0. This property can contain a list of ATOMs,and the each ATOM represents each possible IM Server. IMServer names are restricted to POSIX Portable FilenameCharacter Set. To discover if the IM Server is active, seeif there is an owner for the selection with that atom name.To learn the address of that IM Server, convert theselection target TRANSPORT, which will return a string formof the transport address(es). To learn the supportedlocales of that IM Server, convert the selection targetLOCALES, which will return a set of names of the supportedlocales in the syntax X/Open defines.The basic semantics to determine the IM Server if there aremultiple ATOMs are found in XIM_SERVERS property, is firstfit if the IM Server name is not given as a X modifier’scategory im.The address information retrievable from the TRANSPORTtarget is a transport-specific name. The preregisteredformats for transport-specific names are listed in AppendixB. Additional transport-specific names may be registeredwith X Consortium.For environments that lack X connections, or for IM Serverswhich do not use the X Window System, the preconnectionconvention with IM Server may be given outside the X Windowsystem (e.g. using a Name Service).4. ProtocolThe protocol described below uses the bi-directionalsynchronous/asynchronous request/reply/error model and isspecified using the same conventions outlined in Section 2of the core X Window System protocol [1]:4.1. Basic Requests Packet FormatThis section describes the requests that may be exchangedbetween the client and the IM Server.The basic request packet header format is as follows.major-opcode: CARD8minor-opcode: CARD8length: CARD16The MAJOR-OPCODE specifies which core request or extensionpackage this packet represents. If the MAJOR-OPCODEcorresponds to a core request, the MINOR-OPCODE contains 8bits of request-specific data. (If the MINOR-OPCODE is notused, it is 0.) Otherwise, the MAJOR-OPCODE and theMINOR-OPCODE are specified by XIM_QUERY_EXTENSION message.(Refer to 4.7. Query the supported extension protocol list.)The LENGTH field specifies the number of 4 bytes elementsfollowing the header. If no additional data is followed bythe header, the LENGTH field will be 0.4.2. Data TypesThe following data types are used in the core X IM Serverprotocol:BITMASK16CARD16BITMASK32CARD32PADDING FORMATWhere N is some expression, and Pad(N) is the number of bytes needed to round N up to amultiple of four.Pad(N) = (4 - (N mod 4)) mod 4LPCE1 A character from the4 X Portable Character Set in Latin PortableCharacter Encoding2

X Input Method Protocol X11, Release 6.1

The im-attribute argument specifies XIM values such as XNQueryInputStyle.

XIMTRIGGERKEY

ENCODINGINFO

EXT

XIMATTRIBUTE

XICATTRIBUTE

3

X Input Method Protocol X11, Release 6.1

XIMFEEDBACK

XIMHOTKEYSTATE

XIMPREEDITSTATE

XIMRESETSTATE

4.3. Error NotificationBoth the IM Server and the IM library return XIM_ERRORmessages instead of the corresponding reply messages if anyerrors occur during data processing.At most one error is generated per request. If more than oneerror condition is encountered in processing a request, thechoice of which error is returned isimplementation-dependent.(*1) Before an IM is created, both Input-Method-ID andInput-Context-ID are invalid. Before an IC iscreated, only Input-Method-ID is valid. Afterthat, both of Input-Method-ID and Input-Context-IDare valid.(*2) Unspecific error, for example ‘‘language enginedied’’(*3) This field is reserved for future use.(*4) Vendor defined detail error message4.4. Connection EstablishmentXIM_CONNECT message requests to establish a connection overa mutually-understood virtual stream.(*1) Specify the version of IM Protocol that the clientsupports.A client must send XIM_CONNECT message as the first messageon the connection. The list specifies the names ofauthentication protocols the sending IM Server is willing toperform. (If the client need not authenticate, the list maybe omited.)XIM_AUTH_REQUIRED message is used to send the authenticationprotocol name and protocol-specific data.The auth-protocol is specified by an index into the list ofnames given in the XIM_CONNECT or XIM_AUTH_SETUP message.Any protocol-specific data that might be required is alsosent.The IM library sends XIM_AUTH_REPLY message as the reply toXIM_AUTH_REQUIRED message, if the IM Server isauthenticated.The auth data is specific to the authentication protocol inuse.XIM_AUTH_NEXT message requests to send more auth data.The auth data is specific to the authentication protocol inuse.The IM Server sends XIM_AUTH_SETUP message to authenticatethe client.The list specifies the names of authentication protocols theclient is willing to perform.XIM_AUTH_NG message requests to give up the connection.The IM Server sends XIM_CONNECT_REPLY message as the replyto XIM_CONNECT or XIM_AUTH_REQUIRED message.(*1) Specify the version of IM Protocol that the IMServer supports. This document specifies majorversion one, minor version zero.Here are the state diagrams for the client and the IMServer.State transitions for the clientinit_status:Use authorization function → client_askNot use authorization function → client_no_checkstart:Send XIM_CONNECTIf client_ask → client_wait1If client_no_check,client-auth-protocol-names may be omited →client_wait2client_wait1:Receive XIM_AUTH_REQUIRED → client_checkReceive <other> → client_NGclient_check:If no more auth needed, send XIM_AUTH_REPLY →client_wait2If good auth data, send XIM_AUTH_NEXT →client_wait1If bad auth data, send XIM_AUTH_NG → give up onthis protocolclient_wait2:Receive XIM_CONNECT_REPLY → connectReceive XIM_AUTH_SETUP → client_moreReceive XIM_AUTH_NEXT → client_moreReceive XIM_AUTH_NG → give up on this protocolReceive <other> → client_NGclient_more:Send XIM_AUTH_REQUIRED → client_wait2client_NG:Send XIM_AUTH_NG → give up on this protocolState transitions for the IM Serverinit-status:Use authorization function → server_askNot use authorization function → server_no_checkstart:Receive XIM_CONNECT → start2Receive <other> → server_NGstart2:If client_ask, send XIM_AUTH_REQUIRED →server_wait1If client_no_check and server_ask, sendXIM_AUTH_SETUP → server_wait2If client_no_check and server_no_check, sendXIM_CONNECT_REPLY → connectserver_wait1:Receive XIM_AUTH_REPLY → server2Receive XIM_AUTH_NEXT → server_moreReceive <other> → server_NGserver_moreSend XIM_AUTH_REQUIRED → server_wait1server2If server_ask, send XIM_AUTH_SETUP → server_wait2If server_no_check, send XIM_CONNECT_REPLY →connectserver_wait2Receive XIM_AUTH_REQUIRED → server_checkReceive <other> → server_NGserver_checkIf no more auth data, send XIM_CONNECT_REPLY →connectIf bad auth data, send XIM_AUTH_NG → give up onthis protocolIf good auth data, send XIM_AUTH_NEXT →server_wait2server_NGSend XIM_AUTH_NG → give up on this protocolXIM_DISCONNECT message requests to shutdown the connectionover a mutually-understood virtual stream.XIM_DISCONNECT is a synchronous request. The IM libraryshould wait until it receives either an XIM_DISCONNECT_REPLYpacket or an XIM_ERROR packet.XIM_OPEN requests to establish a logical connection betweenthe IM library and the IM Server.XIM_OPEN is a synchronous request. The IM library shouldwait until receiving either an XIM_OPEN_REPLY packet or anXIM_ERROR packet.XIM_OPEN_REPLY message returns all supported IM and ICattributes in LISTofXIMATTR and LISTofXICATTR. These IM andIC attribute IDs are used to reduce the amount of data whichmust be transferred via the network. In addition, thisindicates to the IM library what kinds of IM/IC attributescan be used in this session, and what types of data will beexchanged. This allows the IM Server provider andapplication writer to support IM system enhancements withnew IM/IC attributes, without modifying Xlib. The IC valuefor the separator of NestedList must be included in theLISTofXICATTR.XIM_CLOSE message requests to shutdown the logicalconnection between the IM library and the IM Server.XIM_CLOSE is a synchronous request. The IM library shouldwait until receiving either an XIM_CLOSE_REPLY packet or anXIM_ERROR packet.4.5. Event Flow ControlAn IM Server must send XIM_SET_EVENT_MASK message to the IMlibrary in order for events to be forwarded to the IMServer, since the IM library initially doesn’t forward anyevents to the IM Server. In the protocol, the IM Server willspecify masks of X events to be forwarded and which need tobe synchronized by the IM library.(*1) Specify all the events to be forwarded to the IMServer by the IM library.(*2) Specify the events to be forwarded withsynchronous flag on by the IM library.XIM_SET_EVENT_MASK is an asynchronous request. The eventmasks are valid immediately after they are set until changedby another XIM_SET_EVENT_MASK message. If input-context-IDis set to zero, the default value of the input-method-IDwill be changed to the event masks specified in the request.That value will be used for the IC’s which have noindividual values.Using the Dynamic Event Flow model, an IM Server sendsXIM_REGISTER_TRIGGERKEYS message to the IM library beforesending XIM_OPEN_REPLY message. Or the IM library maysuppose that the IM Server uses the Static Event Flow model.XIM_REGISTER_TRIGGERKEYS is an asynchronous request. The IMServer notifys the IM library of on-keys and off-keys listswith this message.The IM library notifys the IM Server with XIM_TRIGGER_NOTIFYmessage that a key event matching either on-keys or off-keyshas been occurred.(*1) Specify the events currently selected by the IMlibrary with XSelectInput.XIM_TRIGGER_NOTIFY is a synchronous request. The IM libraryshould wait until receiving either anXIM_TRIGGER_NOTIFY_REPLY packet or an XIM_ERROR packet.4.6. Encoding NegotiationXIM_ENCODING_NEGOTIATION message requests to decide whichencoding to be sent across the wire. When the negotiationfails, the fallback default encoding is Portable CharacterEncoding.The IM Server must choose one encoding from the list sent bythe IM library. If index of the encording determined is -1to indicate that the negotiation is failed, the fallbackdefault encoding is used. The message must be issued aftersending XIM_OPEN message via XOpenIM(). The name ofencoding may be registered with X Consortium.XIM_ENCODING_NEGOTIATION is a synchronous request. The IMlibrary should wait until receiving either anXIM_ENCODING_NEGOTIATION_REPLY packet or an XIM_ERRORpacket.4.7. Query the supported extension protocol listXIM_QUERY_EXTENSION message requests to query the IMextensions supported by the IM Server to which the client isbeing connected.An example of a supported extension is FrontEnd. Themessage must be issued after sending XIM_OPEN message viaXOpenIM().If n is 0, the IM library queries the IM Server for allextensions.If n is not 0, the IM library queries whether the IM Serversupports the contents specified in the list.If a client uses an extension request without previouslyhaving issued a XIM_QUERY_EXTENSION message for thatextension, the IM Server responds with a BadProtocol error.If the IM Server encounters a request with an unknownMAJOR-OPCODE or MINOR-OPCODE, it responds with a BadProtocolerror.XIM_QUERY_EXTENSION is a synchronous request. The IMlibrary should wait until receiving either anXIM_QUERY_EXTENSION_REPLY packet or an XIM_ERROR packet.XIM_QUERY_EXTENSION_REPLY message returns the list ofextensions supported by both the IM library and the IMServer. If the list passed in XIM_QUERY_EXTENSION message isNULL, the IM Server returns the full list of extensionssupported by the IM Server. If the list is not NULL, the IMServer returns the extensions in the list that are supportedby the IM Server.A zero-length string is not a valid extension name. The IMlibrary should disregard any zero-length strings that arereturned in the extension list. The IM library does not usethe requests which are not supported by the IM Server.4.8. Setting IM ValuesXIM_SET_IM_VALUES requests to set attributes to the IM.The im-attributes in XIM_SET_IM_VALUES message are specifiedas a LISTofXIMATTRIBUTE, specifying the attributes to beset. Attributes other than the ones returned byXIM_OPEN_REPLY message should not be specified.XIM_SET_IM_VALUES is a synchronous request. The IM libraryshould wait until receiving either anXIM_SET_IM_VALUES_REPLY packet or an XIM_ERROR packet,because it must receive the error attribute if XIM_ERRORmessage is returned.XIM_SET_IM_VALUES_REPLY message returns the input-method-IDto distinguish replies from multiple IMs.4.9. Getting IM ValuesXIM_GET_IM_VALUES requests to query IM values supported bythe IM Server currently being connected.XIM_GET_IM_VALUES is a synchronous request. The IM libraryshould wait until it receives either anXIM_GET_IM_VALUES_REPLY packet or an XIM_ERROR packet.The IM Server returns IM values with XIM_GET_IM_VALUES_REPLYmessage. The order of the returned im-attribute valuescorresponds directly to that of the list passed with theXIM_GET_IM_VALUES message.4.10. Creating an ICXIM_CREATE_IC message requests to create an IC.The input-context-id is specified by the IM Server toidentify the client (IC). (It is not specified by theclient in XIM_CREATE_IC message.), and it should not be setto zero.XIM_CREATE_IC is a synchronous request which returns theinput-context-ID. The IM library should wait until itreceives either an XIM_CREATE_IC_REPLY packet or anXIM_ERROR packet.4.11. Destroying the ICXIM_DESTROY_IC message requests to destroy the IC.XIM_DESTROY_IC is a synchronous request. The IM libraryshould not free its resources until it receives anXIM_DESTROY_IC_REPLY message because XIM_DESTROY_IC messagemay result in Callback packets such as XIM_PREEDIT_DRAW andXIM_PREEDIT_DONE.4.12. Setting IC ValuesXIM_SET_IC_VALUES messages requests to set attributes to theIC.The ic-attributes in XIM_SET_IC_VALUES message are specifiedas a LISTofXICATTRIBUTE, specifying the attributes to beset. Attributes other than the ones returned byXIM_OPEN_REPLY message should not be specified.XIM_SET_IC_VALUES is a synchronous request. The IM libraryshould wait until receiving either anXIM_SET_IC_VALUES_REPLY packet or an XIM_ERROR packet,because it must receive the error attribute if XIM_ERRORmessage is returned.4.13. Getting IC ValuesXIM_GET_IC_VALUES message requests to query IC valuessupported by the IM Server currently being connected.In LISTofCARD16, the appearance of the ic-attribute-id forthe separator of NestedList shows the end of the headingnested list.XIM_GET_IC_VALUES is a synchronous request and returns eachattribute with its values to show the correspondence. TheIM library should wait until receiving either anXIM_GET_IC_VALUES_REPLY packet or an XIM_ERROR packet.4.14. Setting IC FocusXIM_SET_IC_FOCUS message requests to set the focus to theIC.XIM_SET_IC_FOCUS is an asynchronous request.4.15. Unsetting IC FocusXIM_UNSET_IC_FOCUS message requests to unset the focus tothe focused IC.XIM_UNSET_IC_FOCUS is an asynchronous request.4.16. Filtering EventsEvent filtering is mainly provided for BackEnd method toallow input method to capture X events transparently toclients.X Events are forwarded by XIM_FORWARD_EVENT message. Thismessage can be operated both synchronously andasynchronously. If the requester sets the synchronous flag,the receiver must send XIM_SYNC_REPLY message back to therequester when all the data processing is done.Protocol flow of BackEnd modelWith BackEnd method, the protocol flow can be classifiedinto two methods in terms of synchronization, depending onthe synchronous-eventmask of XIM_SET_EVENT_MASK message.One can be called on-demand-synchronous method and anothercan be called as full-synchronous method.In on-demand-synchronous method, the IM library alwaysreceives XIM_FORWARD_EVENT or XIM_COMMIT message as asynchronous request. Also, the IM Server needs tosynchronously process the correspondent reply from the IMlibrary and the following XIM_FORWARD_EVENT message sentfrom the IM library when any of the event causes the IMServer to send XIM_FORWARD_EVENT or XIM_COMMIT message tothe IM library, so that the input service is consistent. Ifthe IM library gets the control back from the applicationafter receiving the synchronous request, the IM libraryreplies for the synchronous request before processing any ofthe events. In this time, the IM Server blocksXIM_FORWARD_EVENT message which is sent by the IM library,and handles it after receiving the reply. However, the IMServer handles the other protocols at any time.In full-synchronous method, the IM library always sendsXIM_FORWARD_EVENT message to the IM Server as a synchronousrequest. Therefore, the reply to it from the IM Server willbe put between the XIM_FORWARD_EVENT message and itsXIM_SYNC_REPLY message. In case of sendingXIM_FORWARD_EVENT or XIM_COMMIT message, the IM Servershould set the synchronous flag off. Because thesynchronization can be done by the following XIM_SYNC_REPLYmessage.Sample Protocol flow chart 1Following chart shows one of the simplest protocol flowwhich only deals with keyevents for preediting operation.... 0.425 6.888 6.3 10.296 ... 0.000i 3.408i 5.875i 0.000iFig.2 Sample Protocol FlowSample Protocol flow chart 2Following chart shows one of the complex protocol flow,which deals with multiple focus windows and button pressevent as well as keyevent, and the focus is moved by theapplication triggered by both of keyevent and button pressevent. 4

X Input Method Protocol X11, Release 6.1

... 0.425 5.575 6.3 10.296 ... 0.000i 4.721i 5.875i 0.000i

Fig.3 Sample Protocol Flow chart

XEVENT format is same as the X Protocol event format(xEvent). As the value of xEvent’s sequenceNumber is the bottom of 16 bit of XEvent’s xany.serial, the top of 16 bit is sent by serial number(INT16).

XIM_FORWARD_EVENT message is used for forwarding the events from the IM library to the IM Server in order for IM to be able to filter the event. On the other hand, this message is also used for forwarding the events from the IM Server to the IM library if the event forwarded from the IM library is not filtered. The IM Server, which receives XIM_FORWARD_EVENT message without synchronous bit, should set synchronous bit. If both ‘‘request event filtering’’ and ‘‘request lookupstring’’ flag are set, then both filtering and lookup should be done for the same event.

4.17. Synchronizing with the IM ServerXIM_SYNC message requests to synchronize the IM library andthe IM Server.This synchronization can be started either on the IM libraryside or on the IM Server side. The side which receivesXIM_SYNC message should process all XIM requests beforereplying. The input-context-ID is necessary to distinguishthe IC with which the IM library and the IM Server aresynchronized.The side which receives XIM_FORWARD_EVENT, XIM_COMMIT or anyother message with synchronous bit, should process all XIMrequest before replying, and send XIM_SYNC_REPLY message asthe reply to the previous message.4.18. Sending a committed stringWhen the IM Server commits a string, the IM Server sendseither the committed string or list of KeySym, or both, byXIM_COMMIT message.If flag is XLookupKeySym, the arguments continue asfollows:If flag is XLookupChars, the arguments continue asfollows:If flag is XLookupBoth, the arguments continue asfollows:The IM Server which receives XIM_COMMIT message withoutsynchronous bit should set synchronous bit.4.19. Reset ICXIM_RESET_IC message requests to reset the status of IC inthe IM Server.XIM_RESET_IC is a synchronous request. The IM library shouldwait until receiving either an XIM_RESET_IC_REPLY packet oran XIM_ERROR packet.XIM_RESET_IC_REPLY message returns the input-context-ID todistinguish replies from multiple ICs.4.20. CallbacksIf XIMStyle has XIMPreeditArea or XIMStatusArea set,XIMGeometryCallback may be used, and if XIMPreeditCallbackand/or XIMStatusCallback are set, corresponding callbacksmay be used.Any callback request may be sent from an IM Server to an IMclient asynchronously in response to any request previouslysent by the IM client to the IM Server.When an IM Server needs to send a callback requestsynchronously with the request previously sent by an IMclient, the IM Server sends it before replying to theprevious request.4.20.1. Negotiating geometryThe IM Server sends XIM_GEOMETRY message to start geometrynegotiation, if XIMStyle has XIMPreeditArea or XIMStatusAreaset.There is always a single Focus Window, even if some inputfields have only one IC.4.20.2. Converting a stringXIM_STR_CONVERSION message may be used to start the stringconversion from the IM Server.XIM_STR_CONVERSION_REPLY message returns the string to beconverted and the feedback information array.4.20.3. Preedit CallbacksThe IM Server sends XIM_PREEDIT_START message to call theXIMPreeditStartCallback function.The reply to this message must be sent synchronously. Thereply forwards the return value from the callback functionto the IM Server.XIM_PREEDIT_START_REPLY message returns the input-context-IDto distinguish replies from multiple IC’s. The return valuecontains the return value of the functionXIMPreeditStartCallback.The IM Server sends XIM_PREEDIT_DRAW message to call theXIMPreeditDrawCallback function.The fields ‘‘caret’’, ‘‘chg_first’’ and ‘‘chg_length’’correspond to the fields of XIMPreeditDrawCallbackStruct.When the ‘‘no string’’ bit of the status field is set, thetext field of XIMPreeditDrawCallbackStruct is NULL. Whenthe ‘‘no feedback’’ bit of the status field is set, the textfeedback field of XIMPreeditDrawCallbackStruct is NULL.When the above bits are not set, ‘‘preedit string’’ containsthe preedit string to be displayed, and the feedback arraycontains feedback information.The IM Server sends XIM_PREEDIT_CARET message to call thePreeditCaretCallback function.Each entry corresponds to a field ofXIMPreeditCaretCallbackStruct. Since this callback sets thecaret position, its reply must be sent synchronously.The position is the value returned by the callback functionafter it has been called.The IM Server sends XIM_PREEDIT_DONE message to call theXIMPreeditDoneCallback function.4.20.4. Preedit state notifyXIM_PREEDITSTATE message is used to call theXIMPreeditStateNotifyCallback function.4.20.5. Status CallbacksThe IM Server sends XIM_STATUS_START message to call theXIMStatusStartCallback function.XIM_STATUS_START (IM Server → IM library)2 CARD16 input-method-ID2 CARD16 input-context-IDThe IM Server sends XIM_STATUS_DRAW message to call theXIMStatusDrawCallback function.XIM_STATUS_DRAW (IM Server → IM library)2 CARD16 input-method-ID2 CARD16 input-context-ID4 CARD32 type#0 XIMTextType#1 XIMBitmapTypeIf type is XIMTextType, the arguments continue asfollows.4 BITMASK32 status#x0000001 no string#x0000002 no feedback2 n length of status stringn STRING8 status stringp unused, p = Pad(2+n)2 m byte length of feedback array2 unusedm LISTofXIMFEEDBACK feedback arrayIf type is XIMBitmapType, the arguments continue asfollows.4 PIXMAP pixmap dataThe field ‘‘type’’ corresponds to the field inXIMStatusDrawCallbackStruct.The IM Server sends XIM_STATUS_DONE message to call theXIMStatusDoneCallback function.XIM_STATUS_DONE (IM Server → IM library)2 CARD16 input-method-ID2 CARD16 input-context-ID5

X Input Method Protocol X11, Release 6.1

5. AcknowledgementsThis document represents the culmination of several years ofdebate and experiments done under the auspices of the MIT XConsortium i18n working group. Although this was a groupeffort, the author remains responsible for any errors oromissions.We would like to thank to all members of this group. And wewould like to make special thanks to the following people(in alphabetical order) for their participation in the IMProtocol design, Hector Chan, Takashi Fujiwara, YoshioHoriuchi, Makoto Inada, Hiromu Inukai, Mickael Kung, SeijiKuwari, Franky Ling, Hiroyuki Machida, Hiroyuki Miyamoto,Frank Rojas, Bob Scheifler, Makiko Shimamura, ShojiSugiyama, Hidetoshi Tajima, Masaki Takeuchi, MakotoWakamatsu, Masaki Wakao, Nobuyuki Tanaka, Shigeru Yamada,Katsuhisa Yano, Jinsoo Yoon.6. ReferencesAll of the following documents are X Consortium standardsavailable from MIT:[1] Scheifler, Robert W., ‘‘X Window System Protocol Version11’’[2] Scheifler, Robert W. etc., ‘‘Xlib − C Language XInterface’’ 6

X Input Method Protocol X11, Release 6.1

Appendix A

Common Extensions

Extension opcodes and packet names (e.g. XIM_EXT_SET_EVENT_MASK ) for additional extensions may be registered with X Consortium. The following is a commonly well-known extended packet.

XIM_EXT_SET_EVENT_MASK message specifies the set of event masks that the IM library should manipulate.

The IM library must reply XIM_SYNC_REPLY message to the IM Server. This request is valid after the ic is created.

The following requests may be used for improvement of performance.

XIM_EXT_FORWARD_KEYEVENT message may be used instead of XIM_FORWARD_EVENT message.

7

X Input Method Protocol X11, Release 6.1

XIM_EXT_MOVE message may be used to change the spot location instead of XIM_SET_IC_VALUES message. It is effective only if the client specified XIMPreeditPosition.

XIM_EXT_MOVE message is a asynchronous request.

8

X Input Method Protocol X11, Release 6.1

Appendix B

The list of transport specific IM Server address format registered

The following format represents the ATOM contained in XIM_SERVERS property and the string returned from the request converting selection target LOCALES and TRANSPORT.

The following categories are currently registered.

The preregistered formats for transport-specific names are as follows:

If a given category has multiple values, the value is evaluated in order of setting.
9

X Input Method Protocol X11, Release 6.1

Appendix C

Protocol number

Major Protocol number

(*) The IM Server’s extension protocol number should be more than #128.
10

X Input Method Protocol X11, Release 6.1

Appendix D

Implementation Tips

FrontEnd method is recognized as a performance acceleration by the trade off of the variety of the reliability.

In order to use the FrontEnd method, the IM library must query the IM Server to see if the FrontEnd extension is available. The query is made by using the XIM_QUERY_EXTENSION message. The IM Server may send XIM_EXT_SET_EVENT_MASK message with intercept-event-mask, forward-event-mask, and synchronous-event-mask values set after replying XIM_QUERY_EXTENSION_REPLY message.

FrontEnd method can be implemented in a couple of ways depending on how the IM Server utilize XIM_EXT_SET_EVENT_MASK message.

One approach is to update both of the input mask and the filter-event-mask depending on the preeidting state. The sample protocol sequence using the static event flow is as follows:

... 1.675 6.888 6.237 10.296 ... 0.000i 3.408i 4.562i 0.000i │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ _______

To pursuit a maximum performance regardless of the preediting mode, the IM Server may use the dynamic event flow with the following sample protocol sequence.
11

X Input Method Protocol X11, Release 6.1

... 1.675 6.888 6.237 10.296 ... 0.000i 3.408i 4.562i 0.000i │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ _______

This method can reduce the XIM protocol traffic dramatically by updating intercept-event-mask and select-event-mask accordingly. The tradeoff of this performance improvement is that the key events may be lost or disordered in some particular situation, such as when the user types the keyboard in following sequence really fast:

Since this method requires the input mask updates to the both the IM Server and Xlib when turning on and off the preediting, and there is a time lag till the requests take effect when two client issues the input mask updates simultaneously.

Another approach of the FrontEnd method is to update the filter-event-mask depending on the preediting state and not to update the input mask. The IM Server must register both of the preediting on key list and off key list by XIM_REGISTER_TRIGGERKEYS message. In this method, Both the IM Server and the IM client select the same events on the same client’s window, so that the events are delivered to both of the IM Server and the client. The preediting on and off states are expressed by whether the key events are filtered or not. The sample protocol sequence are as follows:

12

X Input Method Protocol X11, Release 6.1

<<Using static event flow>>

... 1.488 7.325 6.487 10.358 ... 0.000i 3.033i 4.999i 0.000i __________ __________ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │

<<Using the dynamic event flow>>

... 1.488 7.325 6.487 10.358 ... 0.000i 3.033i 4.999i 0.000i __________ __________ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │

This method does not have the problem of the time lag when going across the preediting on and off mode, however, the amount of the performance acceleration is not as good as the method described above.

In general, the FrontEnd method requires some synchronization to some of the X protocols, such as the ChangeWindowAttribute protocol for the event mask change or the GrabKey protocol, since it relies on the X’s principal event dispatching mechanism. Any X protocol bindings do not consider the synchronization might cause some mis-synchronization between the IM clients and the IM Server.

13

X Input Method Protocol X11, Release 6.1

The Xlib XIM implementation is layered into three functions, a protocol layer, an interface layer and a transport layer. The purpose of this layering is to make the protocol independent of transport implementation. Each function of these layers are:

The interface layer and the transport layer make various communication channels usable such as X Protocol, TCP/IP, DECnet or STREAM. The following is a sample implementation for the transporter using the X connection. Refer to "xtrans" for the transporter using Socket Transport.

At the beginning of the X Transport connection for the XIM transport mechanism, two different windows must be created either in an Xlib XIM or in an IM Server, with which the Xlib and the IM Server exchange the XIM transports by using the ClientMessage events and Window Properties. In the following, the window created by the Xlib is referred as the "client communication window", and on the other hand, the window created by the IM Server is referred as the "IMS communication window".

Connection

Table D-1; The ClientMessage sent to the IMS window.

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window IMS Window ID
Atom message_type XInternAtom(display, ‘‘_XIM_XCONNECT’’, False)
int format 32
long data.l[0] client communication window ID
long data.l[1] client-major-transport-version (*1)
long data.l[2] client-major-transport-version (*1)

In order to establish the connection (to notify the IM Server communication window), the IM Server sends a ClientMessage in the following event’s format to the client communication window.

14

X Input Method Protocol X11, Release 6.1

Table D-2; The ClientMessage sent by IM Server.

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window client communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_XCONNECT’’, False)
int format 32
long data.l[0] IMS communication window ID
long data.l[1] server-major-transport-version (*1)
long data.l[2] server-minor-transport-version (*1)
long data.l[3] dividing size between ClientMessage and Property (*2)

Table D-3; The read/write method and the major/minor-transport-version

Transport-version read/write
major minor
0 0 only-CM & Property-with-CM
1 only-CM & multi-CM
2 only-CM & multi-CM & Property-with-CM
1 0 PropertyNotify
2 0 only-CM & PropertyNotify
1 only-CM & multi-CM & PropertyNotify
only-CM : data is sent via a ClientMessage
multi-CM : data is sent via multiple ClientMessages
Property-with-CM :
data is written in Property, and its Atom
is send via ClientMessage
PropertyNotify :
data is written in Property, and its Atom
is send via PropertyNotify

read/write

Table D-4; The ClientMessage event’s format (first or middle)

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window IMS communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_MOREDATA’’, False)
int format 8
char data.b[20] (read/write DATA : 20 byte)

Table D-5; The ClientMessage event’s format (only or last)

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window IMS communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_PROTOCOL’’, False)
int format 8
char data.b[20] (read/write DATA : MAX 20 byte) (*1)

15

X Input Method Protocol X11, Release 6.1

Table D-6; The XChangeProperty event’s format

Argument Contents
Display *display The display to which connects
Window window IMS communication window ID
Atom property read/write property Atom (*1)
Atom type XA_STRING
int format 8
int mode PropModeAppend
u_char *data read/write DATA
int nelements length of DATA

Table D-7; The ClientMessage event’s format to send Atom of property

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window IMS communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_PROTOCOL’’, False)
int format 32
long data.l[0] length of read/write property Atom
long data.l[1] read/write property Atom

Table D-8; The ClientMessage event’s format (first or middle)

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window client communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_MOREDATA’’, False)
int format 8
char data.b[20] (read/write DATA : 20 byte)

16

X Input Method Protocol X11, Release 6.1

Table D-9; The ClientMessage event’s format (only or last)

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window client communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_PROTOCOL’’, False)
int format 8
char data.b[20] (read/write DATA : MAX 20 byte) (*1)

Table D-10; The XChangeProperty event’s format

Argument Contents
Display *display The display which to connects
Window window client communication window ID
Atom property read/write property Atom (*1)
Atom type XA_STRING
int format 8
int mode PropModeAppend
u_char *data read/write DATA
int nelements length of DATA

17

X Input Method Protocol X11, Release 6.1

Table D-11; The ClientMessage event’s format to send Atom of property

Structure Member Contents
int type ClientMessage
u_long serial Set by the X Window System
Bool send_event Set by the X Window System
Display *display The display to which connects
Window window client communication window ID
Atom message_type XInternAtom(display, ‘‘_XIM_PROTOCOL’’, False)
int format 32
long data.l[0] length of read/write property ATOM
long data.l[1] read/write property ATOM

Closing Connection

18

X Input Method Protocol X11, Release 6.1

Table of Contents

Appendix B − The list of transport specific IM Serv-