Introduce multi-client IME for special form factors
An advanced multi-display support is requested for certain Android
form-factors so that user(s) can type text on each display at the same
time without losing software keyboard focus in other displays. This is
not possible in existing Android IMEs that are built on top of
InputMethodService class, because the assumption that a single IME
client can be focused at the same time was made before Android IME
APIs were introduced in Android 1.5 and many public APIs in
InputMethodService have already relied heavily on that
assumption. Updating InputMethodService class to support multi-client
scenario is, however, quite challenging because:
1. doing so would introduce an unacceptable amount of complexity into
InputMethodService, which is already hard to maintain,
2. IME developers still need to update their implementation to be
able to support parallel requests from multiple focused IME
client, which may require non-trivial redesign in their side
(e.g. input decoder, typing history database, ...), and
3. actual use cases for multi IME clients are expected to be evolved
rapidly hence the new protocol is not yet stable and not yet ready
to be exposed as public APIs.
This is why a new type of IME needs to be designed and developed
specifically for such special multi-display environments, rather than
reusing existing InputMethodService public class.
Note that there must be no behavior change unless multi-client IME is
explicitly enabled with 'adb shell setprop', which requires root
permission.
See multi-client-ime.md for details.
Fix: 114662040
Test: Manually verified as follows:
1. make -j MultiClientInputMethod
2. adb install -r $OUT/system/priv-app/MultiClientInputMethod/MultiClientInputMethod.apk
3. adb root
4. adb shell setprop persist.debug.multi_client_ime \
com.example.android.multiclientinputmethod/.MultiClientInputMethod
5. adb reboot
6. Try multiple text input scenario
Change-Id: I41dfe854557b178d8af740bc2869c936fc88608b
diff --git a/Android.bp b/Android.bp
index 46fa6e2..092f548 100644
--- a/Android.bp
+++ b/Android.bp
@@ -393,6 +393,9 @@
"core/java/com/android/internal/backup/IObbBackupService.aidl",
"core/java/com/android/internal/inputmethod/IInputContentUriToken.aidl",
"core/java/com/android/internal/inputmethod/IInputMethodPrivilegedOperations.aidl",
+ "core/java/com/android/internal/inputmethod/IMultiClientInputMethod.aidl",
+ "core/java/com/android/internal/inputmethod/IMultiClientInputMethodPrivilegedOperations.aidl",
+ "core/java/com/android/internal/inputmethod/IMultiClientInputMethodSession.aidl",
"core/java/com/android/internal/net/INetworkWatchlistManager.aidl",
"core/java/com/android/internal/policy/IKeyguardDrawnCallback.aidl",
"core/java/com/android/internal/policy/IKeyguardDismissCallback.aidl",
diff --git a/core/java/android/inputmethodservice/MultiClientInputMethodClientCallbackAdaptor.java b/core/java/android/inputmethodservice/MultiClientInputMethodClientCallbackAdaptor.java
new file mode 100644
index 0000000..b4b541d
--- /dev/null
+++ b/core/java/android/inputmethodservice/MultiClientInputMethodClientCallbackAdaptor.java
@@ -0,0 +1,469 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package android.inputmethodservice;
+
+import android.annotation.Nullable;
+import android.annotation.WorkerThread;
+import android.graphics.Rect;
+import android.os.Bundle;
+import android.os.Debug;
+import android.os.Handler;
+import android.os.Looper;
+import android.os.ResultReceiver;
+import android.util.Log;
+import android.view.InputChannel;
+import android.view.InputDevice;
+import android.view.InputEvent;
+import android.view.InputEventReceiver;
+import android.view.KeyEvent;
+import android.view.MotionEvent;
+import android.view.WindowManager.LayoutParams.SoftInputModeFlags;
+import android.view.inputmethod.CompletionInfo;
+import android.view.inputmethod.CursorAnchorInfo;
+import android.view.inputmethod.EditorInfo;
+import android.view.inputmethod.ExtractedText;
+
+import com.android.internal.annotations.GuardedBy;
+import com.android.internal.inputmethod.IMultiClientInputMethodSession;
+import com.android.internal.os.SomeArgs;
+import com.android.internal.util.function.pooled.PooledLambda;
+import com.android.internal.view.IInputContext;
+import com.android.internal.view.IInputMethodSession;
+import com.android.internal.view.InputConnectionWrapper;
+
+import java.lang.ref.WeakReference;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+/**
+ * Re-dispatches all the incoming per-client events to the specified {@link Looper} thread.
+ *
+ * <p>There are three types of per-client callbacks.</p>
+ *
+ * <ul>
+ * <li>{@link IInputMethodSession} - from the IME client</li>
+ * <li>{@link IMultiClientInputMethodSession} - from MultiClientInputMethodManagerService</li>
+ * <li>{@link InputChannel} - from the IME client</li>
+ * </ul>
+ *
+ * <p>This class serializes all the incoming events among those channels onto
+ * {@link MultiClientInputMethodServiceDelegate.ClientCallback} on the specified {@link Looper}
+ * thread.</p>
+ */
+final class MultiClientInputMethodClientCallbackAdaptor {
+ static final boolean DEBUG = false;
+ static final String TAG = MultiClientInputMethodClientCallbackAdaptor.class.getSimpleName();
+
+ private final Object mSessionLock = new Object();
+ @GuardedBy("mSessionLock")
+ CallbackImpl mCallbackImpl;
+ @GuardedBy("mSessionLock")
+ InputChannel mReadChannel;
+ @GuardedBy("mSessionLock")
+ KeyEvent.DispatcherState mDispatcherState;
+ @GuardedBy("mSessionLock")
+ Handler mHandler;
+ @GuardedBy("mSessionLock")
+ @Nullable
+ InputEventReceiver mInputEventReceiver;
+
+ private final AtomicBoolean mFinished = new AtomicBoolean(false);
+
+ IInputMethodSession.Stub createIInputMethodSession() {
+ synchronized (mSessionLock) {
+ return new InputMethodSessionImpl(
+ mSessionLock, mCallbackImpl, mHandler, mFinished);
+ }
+ }
+
+ IMultiClientInputMethodSession.Stub createIMultiClientInputMethodSession() {
+ synchronized (mSessionLock) {
+ return new MultiClientInputMethodSessionImpl(
+ mSessionLock, mCallbackImpl, mHandler, mFinished);
+ }
+ }
+
+ MultiClientInputMethodClientCallbackAdaptor(
+ MultiClientInputMethodServiceDelegate.ClientCallback clientCallback, Looper looper,
+ KeyEvent.DispatcherState dispatcherState, InputChannel readChannel) {
+ synchronized (mSessionLock) {
+ mCallbackImpl = new CallbackImpl(this, clientCallback);
+ mDispatcherState = dispatcherState;
+ mHandler = new Handler(looper, null, true);
+ mReadChannel = readChannel;
+ mInputEventReceiver = new ImeInputEventReceiver(mReadChannel, mHandler.getLooper(),
+ mFinished, mDispatcherState, mCallbackImpl.mOriginalCallback);
+ }
+ }
+
+ private static final class KeyEventCallbackAdaptor implements KeyEvent.Callback {
+ private final MultiClientInputMethodServiceDelegate.ClientCallback mLocalCallback;
+
+ KeyEventCallbackAdaptor(
+ MultiClientInputMethodServiceDelegate.ClientCallback callback) {
+ mLocalCallback = callback;
+ }
+
+ @Override
+ public boolean onKeyDown(int keyCode, KeyEvent event) {
+ return mLocalCallback.onKeyDown(keyCode, event);
+ }
+
+ @Override
+ public boolean onKeyLongPress(int keyCode, KeyEvent event) {
+ return mLocalCallback.onKeyLongPress(keyCode, event);
+ }
+
+ @Override
+ public boolean onKeyUp(int keyCode, KeyEvent event) {
+ return mLocalCallback.onKeyUp(keyCode, event);
+ }
+
+ @Override
+ public boolean onKeyMultiple(int keyCode, int count, KeyEvent event) {
+ return mLocalCallback.onKeyMultiple(keyCode, event);
+ }
+ }
+
+ private static final class ImeInputEventReceiver extends InputEventReceiver {
+ private final AtomicBoolean mFinished;
+ private final KeyEvent.DispatcherState mDispatcherState;
+ private final MultiClientInputMethodServiceDelegate.ClientCallback mClientCallback;
+ private final KeyEventCallbackAdaptor mKeyEventCallbackAdaptor;
+
+ ImeInputEventReceiver(InputChannel readChannel, Looper looper, AtomicBoolean finished,
+ KeyEvent.DispatcherState dispatcherState,
+ MultiClientInputMethodServiceDelegate.ClientCallback callback) {
+ super(readChannel, looper);
+ mFinished = finished;
+ mDispatcherState = dispatcherState;
+ mClientCallback = callback;
+ mKeyEventCallbackAdaptor = new KeyEventCallbackAdaptor(callback);
+ }
+
+ @Override
+ public void onInputEvent(InputEvent event) {
+ if (mFinished.get()) {
+ // The session has been finished.
+ finishInputEvent(event, false);
+ return;
+ }
+ boolean handled = false;
+ try {
+ if (event instanceof KeyEvent) {
+ final KeyEvent keyEvent = (KeyEvent) event;
+ handled = keyEvent.dispatch(mKeyEventCallbackAdaptor, mDispatcherState,
+ mKeyEventCallbackAdaptor);
+ } else {
+ final MotionEvent motionEvent = (MotionEvent) event;
+ if (motionEvent.isFromSource(InputDevice.SOURCE_CLASS_TRACKBALL)) {
+ handled = mClientCallback.onTrackballEvent(motionEvent);
+ } else {
+ handled = mClientCallback.onGenericMotionEvent(motionEvent);
+ }
+ }
+ } finally {
+ finishInputEvent(event, handled);
+ }
+ }
+ }
+
+ private static final class InputMethodSessionImpl extends IInputMethodSession.Stub {
+ private final Object mSessionLock;
+ @GuardedBy("mSessionLock")
+ private CallbackImpl mCallbackImpl;
+ @GuardedBy("mSessionLock")
+ private Handler mHandler;
+ private final AtomicBoolean mSessionFinished;
+
+ InputMethodSessionImpl(Object lock, CallbackImpl callback, Handler handler,
+ AtomicBoolean sessionFinished) {
+ mSessionLock = lock;
+ mCallbackImpl = callback;
+ mHandler = handler;
+ mSessionFinished = sessionFinished;
+ }
+
+ @Override
+ public void updateExtractedText(int token, ExtractedText text) {
+ reportNotSupported();
+ }
+
+ @Override
+ public void updateSelection(int oldSelStart, int oldSelEnd,
+ int newSelStart, int newSelEnd,
+ int candidatesStart, int candidatesEnd) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ final SomeArgs args = SomeArgs.obtain();
+ args.argi1 = oldSelStart;
+ args.argi2 = oldSelEnd;
+ args.argi3 = newSelStart;
+ args.argi4 = newSelEnd;
+ args.argi5 = candidatesStart;
+ args.argi6 = candidatesEnd;
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::updateSelection, mCallbackImpl, args));
+ }
+ }
+
+ @Override
+ public void viewClicked(boolean focusChanged) {
+ reportNotSupported();
+ }
+
+ @Override
+ public void updateCursor(Rect newCursor) {
+ reportNotSupported();
+ }
+
+ @Override
+ public void displayCompletions(CompletionInfo[] completions) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::displayCompletions, mCallbackImpl, completions));
+ }
+ }
+
+ @Override
+ public void appPrivateCommand(String action, Bundle data) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::appPrivateCommand, mCallbackImpl, action, data));
+ }
+ }
+
+ @Override
+ public void toggleSoftInput(int showFlags, int hideFlags) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::toggleSoftInput, mCallbackImpl, showFlags,
+ hideFlags));
+ }
+ }
+
+ @Override
+ public void finishSession() {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mSessionFinished.set(true);
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::finishSession, mCallbackImpl));
+ mCallbackImpl = null;
+ mHandler = null;
+ }
+ }
+
+ @Override
+ public void updateCursorAnchorInfo(CursorAnchorInfo info) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::updateCursorAnchorInfo, mCallbackImpl, info));
+ }
+ }
+ }
+
+ private static final class MultiClientInputMethodSessionImpl
+ extends IMultiClientInputMethodSession.Stub {
+ private final Object mSessionLock;
+ @GuardedBy("mSessionLock")
+ private CallbackImpl mCallbackImpl;
+ @GuardedBy("mSessionLock")
+ private Handler mHandler;
+ private final AtomicBoolean mSessionFinished;
+
+ MultiClientInputMethodSessionImpl(Object lock, CallbackImpl callback,
+ Handler handler, AtomicBoolean sessionFinished) {
+ mSessionLock = lock;
+ mCallbackImpl = callback;
+ mHandler = handler;
+ mSessionFinished = sessionFinished;
+ }
+
+ @Override
+ public void startInputOrWindowGainedFocus(@Nullable IInputContext inputContext,
+ int missingMethods, @Nullable EditorInfo editorInfo, int controlFlags,
+ @SoftInputModeFlags int softInputMode, int windowHandle) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ final SomeArgs args = SomeArgs.obtain();
+ // TODO(Bug 119211536): Remove dependency on AbstractInputMethodService from ICW
+ final WeakReference<AbstractInputMethodService> fakeIMS =
+ new WeakReference<>(null);
+ args.arg1 = (inputContext == null) ? null
+ : new InputConnectionWrapper(fakeIMS, inputContext, missingMethods,
+ mSessionFinished);
+ args.arg2 = editorInfo;
+ args.argi1 = controlFlags;
+ args.argi2 = softInputMode;
+ args.argi3 = windowHandle;
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::startInputOrWindowGainedFocus, mCallbackImpl, args));
+ }
+ }
+
+ @Override
+ public void showSoftInput(int flags, ResultReceiver resultReceiver) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::showSoftInput, mCallbackImpl, flags,
+ resultReceiver));
+ }
+ }
+
+ @Override
+ public void hideSoftInput(int flags, ResultReceiver resultReceiver) {
+ synchronized (mSessionLock) {
+ if (mCallbackImpl == null || mHandler == null) {
+ return;
+ }
+ mHandler.sendMessage(PooledLambda.obtainMessage(
+ CallbackImpl::hideSoftInput, mCallbackImpl, flags,
+ resultReceiver));
+ }
+ }
+ }
+
+ /**
+ * The maim part of adaptor to {@link MultiClientInputMethodServiceDelegate.ClientCallback}.
+ */
+ @WorkerThread
+ private static final class CallbackImpl {
+ private final MultiClientInputMethodClientCallbackAdaptor mCallbackAdaptor;
+ private final MultiClientInputMethodServiceDelegate.ClientCallback mOriginalCallback;
+ private boolean mFinished = false;
+
+ CallbackImpl(MultiClientInputMethodClientCallbackAdaptor callbackAdaptor,
+ MultiClientInputMethodServiceDelegate.ClientCallback callback) {
+ mCallbackAdaptor = callbackAdaptor;
+ mOriginalCallback = callback;
+ }
+
+ void updateSelection(SomeArgs args) {
+ try {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onUpdateSelection(args.argi1, args.argi2, args.argi3,
+ args.argi4, args.argi5, args.argi6);
+ } finally {
+ args.recycle();
+ }
+ }
+
+ void displayCompletions(CompletionInfo[] completions) {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onDisplayCompletions(completions);
+ }
+
+ void appPrivateCommand(String action, Bundle data) {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onAppPrivateCommand(action, data);
+ }
+
+ void toggleSoftInput(int showFlags, int hideFlags) {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onToggleSoftInput(showFlags, hideFlags);
+ }
+
+ void finishSession() {
+ if (mFinished) {
+ return;
+ }
+ mFinished = true;
+ mOriginalCallback.onFinishSession();
+ synchronized (mCallbackAdaptor.mSessionLock) {
+ mCallbackAdaptor.mDispatcherState = null;
+ if (mCallbackAdaptor.mReadChannel != null) {
+ mCallbackAdaptor.mReadChannel.dispose();
+ mCallbackAdaptor.mReadChannel = null;
+ }
+ mCallbackAdaptor.mInputEventReceiver = null;
+ }
+ }
+
+ void updateCursorAnchorInfo(CursorAnchorInfo info) {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onUpdateCursorAnchorInfo(info);
+ }
+
+ void startInputOrWindowGainedFocus(SomeArgs args) {
+ try {
+ if (mFinished) {
+ return;
+ }
+ final InputConnectionWrapper inputConnection = (InputConnectionWrapper) args.arg1;
+ final EditorInfo editorInfo = (EditorInfo) args.arg2;
+ final int startInputFlags = args.argi1;
+ final int softInputMode = args.argi2;
+ final int windowHandle = args.argi3;
+ mOriginalCallback.onStartInputOrWindowGainedFocus(inputConnection, editorInfo,
+ startInputFlags, softInputMode, windowHandle);
+ } finally {
+ args.recycle();
+ }
+ }
+
+ void showSoftInput(int flags, ResultReceiver resultReceiver) {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onShowSoftInput(flags, resultReceiver);
+ }
+
+ void hideSoftInput(int flags, ResultReceiver resultReceiver) {
+ if (mFinished) {
+ return;
+ }
+ mOriginalCallback.onHideSoftInput(flags, resultReceiver);
+ }
+ }
+
+ private static void reportNotSupported() {
+ if (DEBUG) {
+ Log.d(TAG, Debug.getCaller() + " is not supported");
+ }
+ }
+}
diff --git a/core/java/android/inputmethodservice/MultiClientInputMethodServiceDelegate.java b/core/java/android/inputmethodservice/MultiClientInputMethodServiceDelegate.java
new file mode 100644
index 0000000..0604f6a6
--- /dev/null
+++ b/core/java/android/inputmethodservice/MultiClientInputMethodServiceDelegate.java
@@ -0,0 +1,377 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package android.inputmethodservice;
+
+import android.annotation.Nullable;
+import android.content.Context;
+import android.content.Intent;
+import android.os.Bundle;
+import android.os.IBinder;
+import android.os.Looper;
+import android.os.ResultReceiver;
+import android.view.KeyEvent;
+import android.view.MotionEvent;
+import android.view.View;
+import android.view.WindowManager.LayoutParams.SoftInputModeFlags;
+import android.view.inputmethod.CompletionInfo;
+import android.view.inputmethod.CursorAnchorInfo;
+import android.view.inputmethod.EditorInfo;
+import android.view.inputmethod.InputConnection;
+
+import com.android.internal.inputmethod.StartInputFlags;
+
+/**
+ * Defines all the public APIs and interfaces that are necessary to implement multi-client IMEs.
+ *
+ * <p>Actual implementation is further delegated to
+ * {@link MultiClientInputMethodServiceDelegateImpl}.</p>
+ *
+ * @hide
+ */
+public final class MultiClientInputMethodServiceDelegate {
+ // @SdkConstant(SdkConstantType.SERVICE_ACTION)
+ public static final String SERVICE_INTERFACE =
+ "android.inputmethodservice.MultiClientInputMethodService";
+
+ /**
+ * Special value that is guaranteed to be not used for IME client ID.
+ */
+ public static final int INVALID_CLIENT_ID = -1;
+
+ /**
+ * Special value that is guaranteed to be not used for window handle.
+ */
+ public static final int INVALID_WINDOW_HANDLE = -1;
+
+ private final MultiClientInputMethodServiceDelegateImpl mImpl;
+
+ /**
+ * Top-level callbacks for this {@link MultiClientInputMethodServiceDelegate}.
+ */
+ public interface ServiceCallback {
+ /**
+ * Called when this {@link MultiClientInputMethodServiceDelegate} is recognized by the
+ * system and privileged operations like {@link #createInputMethodWindowToken(int)} are
+ * ready to be called.
+ */
+ void initialized();
+
+ /**
+ * Called when a new IME client is recognized by the system.
+ *
+ * <p>Once the IME receives this callback, the IME can start interacting with the IME client
+ * by calling {@link #acceptClient(int, ClientCallback, KeyEvent.DispatcherState, Looper)}.
+ * </p>
+ *
+ * @param clientId ID of the client.
+ * @param uid UID of the IME client.
+ * @param pid PID of the IME client.
+ * @param selfReportedDisplayId display ID reported from the IME client. Since the system
+ * does not validate this display ID, and at any time the IME client can lose the
+ * access to this display ID, the IME needs to call
+ * {@link #isUidAllowedOnDisplay(int, int)} to check whether the IME client still
+ * has access to this display or not.
+ */
+ void addClient(int clientId, int uid, int pid, int selfReportedDisplayId);
+
+ /**
+ * Called when an IME client is being destroyed.
+ *
+ * @param clientId ID of the client.
+ */
+ void removeClient(int clientId);
+ }
+
+ /**
+ * Per-client callbacks.
+ */
+ public interface ClientCallback {
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#sendAppPrivateCommand(View, String, Bundle)}.
+ *
+ * @param action Name of the command to be performed.
+ * @param data Any data to include with the command.
+ * @see android.inputmethodservice.InputMethodService#onAppPrivateCommand(String, Bundle)
+ */
+ void onAppPrivateCommand(String action, Bundle data);
+
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#displayCompletions(View, CompletionInfo[])}.
+ *
+ * @param completions Completion information provided from the IME client.
+ * @see android.inputmethodservice.InputMethodService#onDisplayCompletions(CompletionInfo[])
+ */
+ void onDisplayCompletions(CompletionInfo[] completions);
+
+ /**
+ * Called when this callback session is closed. No further callback should not happen on
+ * this callback object.
+ */
+ void onFinishSession();
+
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#hideSoftInputFromWindow(IBinder, int)} or
+ * {@link android.view.inputmethod.InputMethodManager#hideSoftInputFromWindow(IBinder, int,
+ * ResultReceiver)}.
+ *
+ * @param flags The flag passed by the client.
+ * @param resultReceiver The {@link ResultReceiver} passed by the client.
+ * @see android.inputmethodservice.InputMethodService#onWindowHidden()
+ */
+ void onHideSoftInput(int flags, ResultReceiver resultReceiver);
+
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#showSoftInput(View, int)} or {@link
+ * android.view.inputmethod.InputMethodManager#showSoftInput(View, int, ResultReceiver)}.
+ *
+ * @param flags The flag passed by the client.
+ * @param resultReceiver The {@link ResultReceiver} passed by the client.
+ * @see android.inputmethodservice.InputMethodService#onWindowShown()
+ */
+ void onShowSoftInput(int flags, ResultReceiver resultReceiver);
+
+ /**
+ * A generic callback when {@link InputConnection} is being established.
+ *
+ * @param inputConnection The {@link InputConnection} to be established.
+ * @param editorInfo The {@link EditorInfo} reported from the IME client.
+ * @param startInputFlags Any combinations of {@link StartInputFlags}.
+ * @param softInputMode SoftWindowMode specified to this window.
+ * @param targetWindowHandle A unique Window token.
+ * @see android.inputmethodservice.InputMethodService#onStartInput(EditorInfo, boolean)
+ */
+ void onStartInputOrWindowGainedFocus(
+ @Nullable InputConnection inputConnection,
+ @Nullable EditorInfo editorInfo,
+ @StartInputFlags int startInputFlags,
+ @SoftInputModeFlags int softInputMode,
+ int targetWindowHandle);
+
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#toggleSoftInput(int, int)}.
+ *
+ * @param showFlags The flag passed by the client.
+ * @param hideFlags The flag passed by the client.
+ * @see android.inputmethodservice.InputMethodService#onToggleSoftInput(int, int)
+ */
+ void onToggleSoftInput(int showFlags, int hideFlags);
+
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#updateCursorAnchorInfo(View,
+ * CursorAnchorInfo)}.
+ *
+ * @param info The {@link CursorAnchorInfo} passed by the client.
+ * @see android.inputmethodservice.InputMethodService#onUpdateCursorAnchorInfo(
+ * CursorAnchorInfo)
+ */
+ void onUpdateCursorAnchorInfo(CursorAnchorInfo info);
+
+ /**
+ * Called when the associated IME client called {@link
+ * android.view.inputmethod.InputMethodManager#updateSelection(View, int, int, int, int)}.
+ *
+ * @param oldSelStart The previous selection start index.
+ * @param oldSelEnd The previous selection end index.
+ * @param newSelStart The new selection start index.
+ * @param newSelEnd The new selection end index.
+ * @param candidatesStart The new candidate start index.
+ * @param candidatesEnd The new candidate end index.
+ * @see android.inputmethodservice.InputMethodService#onUpdateSelection(int, int, int, int,
+ * int, int)
+ */
+ void onUpdateSelection(int oldSelStart, int oldSelEnd, int newSelStart, int newSelEnd,
+ int candidatesStart, int candidatesEnd);
+
+ /**
+ * Called to give a chance for the IME to intercept generic motion events before they are
+ * processed by the application.
+ *
+ * @param event {@link MotionEvent} that is about to be handled by the IME client.
+ * @return {@code true} to tell the IME client that the IME handled this event.
+ * @see android.inputmethodservice.InputMethodService#onGenericMotionEvent(MotionEvent)
+ */
+ boolean onGenericMotionEvent(MotionEvent event);
+
+ /**
+ * Called to give a chance for the IME to intercept key down events before they are
+ * processed by the application.
+ *
+ * @param keyCode The value in {@link KeyEvent#getKeyCode()}.
+ * @param event {@link KeyEvent} for this key down event.
+ * @return {@code true} to tell the IME client that the IME handled this event.
+ * @see android.inputmethodservice.InputMethodService#onKeyDown(int, KeyEvent)
+ */
+ boolean onKeyDown(int keyCode, KeyEvent event);
+
+ /**
+ * Called to give a chance for the IME to intercept key long press events before they are
+ * processed by the application.
+ *
+ * @param keyCode The value in {@link KeyEvent#getKeyCode()}.
+ * @param event {@link KeyEvent} for this key long press event.
+ * @return {@code true} to tell the IME client that the IME handled this event.
+ * @see android.inputmethodservice.InputMethodService#onKeyLongPress(int, KeyEvent)
+ */
+ boolean onKeyLongPress(int keyCode, KeyEvent event);
+
+ /**
+ * Called to give a chance for the IME to intercept key multiple events before they are
+ * processed by the application.
+ *
+ * @param keyCode The value in {@link KeyEvent#getKeyCode()}.
+ * @param event {@link KeyEvent} for this key multiple event.
+ * @return {@code true} to tell the IME client that the IME handled this event.
+ * @see android.inputmethodservice.InputMethodService#onKeyMultiple(int, int, KeyEvent)
+ */
+ boolean onKeyMultiple(int keyCode, KeyEvent event);
+
+ /**
+ * Called to give a chance for the IME to intercept key up events before they are processed
+ * by the application.
+ *
+ * @param keyCode The value in {@link KeyEvent#getKeyCode()}.
+ * @param event {@link KeyEvent} for this key up event.
+ * @return {@code true} to tell the IME client that the IME handled this event.
+ * @see android.inputmethodservice.InputMethodService#onKeyUp(int, KeyEvent)
+ */
+ boolean onKeyUp(int keyCode, KeyEvent event);
+
+ /**
+ * Called to give a chance for the IME to intercept generic motion events before they are
+ * processed by the application.
+ *
+ * @param event {@link MotionEvent} that is about to be handled by the IME client.
+ * @return {@code true} to tell the IME client that the IME handled this event.
+ * @see android.inputmethodservice.InputMethodService#onTrackballEvent(MotionEvent)
+ */
+ boolean onTrackballEvent(MotionEvent event);
+ }
+
+ private MultiClientInputMethodServiceDelegate(Context context,
+ ServiceCallback serviceCallback) {
+ mImpl = new MultiClientInputMethodServiceDelegateImpl(context, serviceCallback);
+ }
+
+ /**
+ * Must be called by the multi-client IME implementer to create
+ * {@link MultiClientInputMethodServiceDelegate}.
+ *
+ * @param context {@link Context} with which the delegate should interact with the system.
+ * @param serviceCallback {@link ServiceCallback} to receive service-level callbacks.
+ * @return A new instance of {@link MultiClientInputMethodServiceDelegate}.
+ */
+ public static MultiClientInputMethodServiceDelegate create(Context context,
+ ServiceCallback serviceCallback) {
+ return new MultiClientInputMethodServiceDelegate(context, serviceCallback);
+ }
+
+ /**
+ * Must be called by the multi-client IME service when {@link android.app.Service#onDestroy()}
+ * is called.
+ */
+ public void onDestroy() {
+ mImpl.onDestroy();
+ }
+
+ /**
+ * Must be called by the multi-client IME service when
+ * {@link android.app.Service#onBind(Intent)} is called.
+ *
+ * @param intent {@link Intent} passed to {@link android.app.Service#onBind(Intent)}.
+ * @return An {@link IBinder} object that needs to be returned from
+ * {@link android.app.Service#onBind(Intent)}.
+ */
+ public IBinder onBind(Intent intent) {
+ return mImpl.onBind(intent);
+ }
+
+ /**
+ * Must be called by the multi-client IME service when
+ * {@link android.app.Service#onUnbind(Intent)} is called.
+ *
+ * @param intent {@link Intent} passed to {@link android.app.Service#onUnbind(Intent)}.
+ * @return A boolean value that needs to be returned from
+ * {@link android.app.Service#onUnbind(Intent)}.
+ */
+ public boolean onUnbind(Intent intent) {
+ return mImpl.onUnbind(intent);
+ }
+
+ /**
+ * Must be called by the multi-client IME service to create a special window token for IME
+ * window.
+ *
+ * <p>This method is available only after {@link ServiceCallback#initialized()}.</p>
+ *
+ * @param displayId display ID on which the IME window will be shown.
+ * @return Window token to be specified to the IME window/
+ */
+ public IBinder createInputMethodWindowToken(int displayId) {
+ return mImpl.createInputMethodWindowToken(displayId);
+ }
+
+ /**
+ * Must be called by the multi-client IME service to notify the system when the IME is ready to
+ * accept callback events from the specified IME client.
+ *
+ * @param clientId The IME client ID specified in
+ * {@link ServiceCallback#addClient(int, int, int, int)}.
+ * @param clientCallback The {@link ClientCallback} to receive callback events from this IME
+ * client.
+ * @param dispatcherState {@link KeyEvent.DispatcherState} to be used when receiving key-related
+ * callbacks in {@link ClientCallback}.
+ * @param looper {@link Looper} on which {@link ClientCallback} will be called back.
+ */
+ public void acceptClient(int clientId, ClientCallback clientCallback,
+ KeyEvent.DispatcherState dispatcherState, Looper looper) {
+ mImpl.acceptClient(clientId, clientCallback, dispatcherState, looper);
+ }
+
+ /**
+ * Must be called by the multi-client IME service to notify the system when the IME is ready to
+ * interact with the window in the IME client.
+ *
+ * @param clientId The IME client ID specified in
+ * {@link ServiceCallback#addClient(int, int, int, int)}.
+ * @param targetWindowHandle The window handle specified in
+ * {@link ClientCallback#onStartInputOrWindowGainedFocus}.
+ * @param imeWindowToken The IME window token returned from
+ * {@link #createInputMethodWindowToken(int)}.
+ */
+ public void reportImeWindowTarget(int clientId, int targetWindowHandle,
+ IBinder imeWindowToken) {
+ mImpl.reportImeWindowTarget(clientId, targetWindowHandle, imeWindowToken);
+ }
+
+ /**
+ * Can be called by the multi-client IME service to check if the given {@code uid} is allowed
+ * to access to {@code displayId}.
+ *
+ * @param displayId Display ID to be queried.
+ * @param uid UID to be queried.
+ * @return {@code true} if {@code uid} is allowed to access to {@code displayId}.
+ */
+ public boolean isUidAllowedOnDisplay(int displayId, int uid) {
+ return mImpl.isUidAllowedOnDisplay(displayId, uid);
+ }
+}
diff --git a/core/java/android/inputmethodservice/MultiClientInputMethodServiceDelegateImpl.java b/core/java/android/inputmethodservice/MultiClientInputMethodServiceDelegateImpl.java
new file mode 100644
index 0000000..bbe3a7f
--- /dev/null
+++ b/core/java/android/inputmethodservice/MultiClientInputMethodServiceDelegateImpl.java
@@ -0,0 +1,193 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package android.inputmethodservice;
+
+import static java.lang.annotation.RetentionPolicy.SOURCE;
+
+import android.annotation.IntDef;
+import android.content.Context;
+import android.content.Intent;
+import android.os.IBinder;
+import android.os.Looper;
+import android.util.Log;
+import android.view.InputChannel;
+import android.view.KeyEvent;
+
+import com.android.internal.annotations.GuardedBy;
+import com.android.internal.inputmethod.IMultiClientInputMethod;
+import com.android.internal.inputmethod.IMultiClientInputMethodPrivilegedOperations;
+import com.android.internal.inputmethod.MultiClientInputMethodPrivilegedOperations;
+
+import java.lang.annotation.Retention;
+import java.lang.ref.WeakReference;
+
+final class MultiClientInputMethodServiceDelegateImpl {
+ private static final String TAG = "MultiClientInputMethodServiceDelegateImpl";
+
+ private final Object mLock = new Object();
+
+ @Retention(SOURCE)
+ @IntDef({InitializationPhase.INSTANTIATED,
+ InitializationPhase.ON_BIND_CALLED,
+ InitializationPhase.INITIALIZE_CALLED,
+ InitializationPhase.ON_UNBIND_CALLED,
+ InitializationPhase.ON_DESTROY_CALLED})
+ private @interface InitializationPhase {
+ int INSTANTIATED = 1;
+ int ON_BIND_CALLED = 2;
+ int INITIALIZE_CALLED = 3;
+ int ON_UNBIND_CALLED = 4;
+ int ON_DESTROY_CALLED = 5;
+ }
+
+ @GuardedBy("mLock")
+ @InitializationPhase
+ private int mInitializationPhase;
+
+ private final MultiClientInputMethodPrivilegedOperations mPrivOps =
+ new MultiClientInputMethodPrivilegedOperations();
+
+ private final MultiClientInputMethodServiceDelegate.ServiceCallback mServiceCallback;
+
+ private final Context mContext;
+
+ MultiClientInputMethodServiceDelegateImpl(Context context,
+ MultiClientInputMethodServiceDelegate.ServiceCallback serviceCallback) {
+ mInitializationPhase = InitializationPhase.INSTANTIATED;
+ mContext = context;
+ mServiceCallback = serviceCallback;
+ }
+
+ void onDestroy() {
+ synchronized (mLock) {
+ switch (mInitializationPhase) {
+ case InitializationPhase.INSTANTIATED:
+ case InitializationPhase.ON_UNBIND_CALLED:
+ mInitializationPhase = InitializationPhase.ON_DESTROY_CALLED;
+ break;
+ default:
+ Log.e(TAG, "unexpected state=" + mInitializationPhase);
+ break;
+ }
+ }
+ }
+
+ private static final class ServiceImpl extends IMultiClientInputMethod.Stub {
+ private final WeakReference<MultiClientInputMethodServiceDelegateImpl> mImpl;
+
+ ServiceImpl(MultiClientInputMethodServiceDelegateImpl service) {
+ mImpl = new WeakReference<>(service);
+ }
+
+ @Override
+ public void initialize(IMultiClientInputMethodPrivilegedOperations privOps) {
+ final MultiClientInputMethodServiceDelegateImpl service = mImpl.get();
+ if (service == null) {
+ return;
+ }
+ synchronized (service.mLock) {
+ switch (service.mInitializationPhase) {
+ case InitializationPhase.ON_BIND_CALLED:
+ service.mPrivOps.set(privOps);
+ service.mInitializationPhase = InitializationPhase.INITIALIZE_CALLED;
+ service.mServiceCallback.initialized();
+ break;
+ default:
+ Log.e(TAG, "unexpected state=" + service.mInitializationPhase);
+ break;
+ }
+ }
+ }
+
+ @Override
+ public void addClient(int clientId, int uid, int pid, int selfReportedDisplayId) {
+ final MultiClientInputMethodServiceDelegateImpl service = mImpl.get();
+ if (service == null) {
+ return;
+ }
+ service.mServiceCallback.addClient(clientId, uid, pid, selfReportedDisplayId);
+ }
+
+ @Override
+ public void removeClient(int clientId) {
+ final MultiClientInputMethodServiceDelegateImpl service = mImpl.get();
+ if (service == null) {
+ return;
+ }
+ service.mServiceCallback.removeClient(clientId);
+ }
+ }
+
+ IBinder onBind(Intent intent) {
+ synchronized (mLock) {
+ switch (mInitializationPhase) {
+ case InitializationPhase.INSTANTIATED:
+ mInitializationPhase = InitializationPhase.ON_BIND_CALLED;
+ return new ServiceImpl(this);
+ default:
+ Log.e(TAG, "unexpected state=" + mInitializationPhase);
+ break;
+ }
+ }
+ return null;
+ }
+
+ boolean onUnbind(Intent intent) {
+ synchronized (mLock) {
+ switch (mInitializationPhase) {
+ case InitializationPhase.ON_BIND_CALLED:
+ case InitializationPhase.INITIALIZE_CALLED:
+ mInitializationPhase = InitializationPhase.ON_UNBIND_CALLED;
+ mPrivOps.dispose();
+ break;
+ default:
+ Log.e(TAG, "unexpected state=" + mInitializationPhase);
+ break;
+ }
+ }
+ return false;
+ }
+
+ IBinder createInputMethodWindowToken(int displayId) {
+ return mPrivOps.createInputMethodWindowToken(displayId);
+ }
+
+ void acceptClient(int clientId,
+ MultiClientInputMethodServiceDelegate.ClientCallback clientCallback,
+ KeyEvent.DispatcherState dispatcherState, Looper looper) {
+ final InputChannel[] channels = InputChannel.openInputChannelPair("MSIMS-session");
+ final InputChannel writeChannel = channels[0];
+ final InputChannel readChannel = channels[1];
+ try {
+ final MultiClientInputMethodClientCallbackAdaptor callbackAdaptor =
+ new MultiClientInputMethodClientCallbackAdaptor(clientCallback, looper,
+ dispatcherState, readChannel);
+ mPrivOps.acceptClient(clientId, callbackAdaptor.createIInputMethodSession(),
+ callbackAdaptor.createIMultiClientInputMethodSession(), writeChannel);
+ } finally {
+ writeChannel.dispose();
+ }
+ }
+
+ void reportImeWindowTarget(int clientId, int targetWindowHandle, IBinder imeWindowToken) {
+ mPrivOps.reportImeWindowTarget(clientId, targetWindowHandle, imeWindowToken);
+ }
+
+ boolean isUidAllowedOnDisplay(int displayId, int uid) {
+ return mPrivOps.isUidAllowedOnDisplay(displayId, uid);
+ }
+}
diff --git a/core/java/com/android/internal/inputmethod/IMultiClientInputMethod.aidl b/core/java/com/android/internal/inputmethod/IMultiClientInputMethod.aidl
new file mode 100644
index 0000000..2f782ba
--- /dev/null
+++ b/core/java/com/android/internal/inputmethod/IMultiClientInputMethod.aidl
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.android.internal.inputmethod;
+
+import android.os.IBinder;
+import com.android.internal.inputmethod.IMultiClientInputMethodPrivilegedOperations;
+
+oneway interface IMultiClientInputMethod {
+ void initialize(in IMultiClientInputMethodPrivilegedOperations privOps);
+ void addClient(int clientId, int uid, int pid, int selfReportedDisplayId);
+ void removeClient(int clientId);
+}
diff --git a/core/java/com/android/internal/inputmethod/IMultiClientInputMethodPrivilegedOperations.aidl b/core/java/com/android/internal/inputmethod/IMultiClientInputMethodPrivilegedOperations.aidl
new file mode 100644
index 0000000..69d9ccc
--- /dev/null
+++ b/core/java/com/android/internal/inputmethod/IMultiClientInputMethodPrivilegedOperations.aidl
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.android.internal.inputmethod;
+
+import android.view.InputChannel;
+import com.android.internal.view.IInputMethodSession;
+import com.android.internal.inputmethod.IMultiClientInputMethodSession;
+
+/**
+ * Defines priviledged operations that only the current MSIMS is allowed to call.
+ * Actual operations are implemented and handled by MultiClientInputMethodManagerService.
+ */
+interface IMultiClientInputMethodPrivilegedOperations {
+ IBinder createInputMethodWindowToken(int displayId);
+ void deleteInputMethodWindowToken(IBinder token);
+ void acceptClient(int clientId, in IInputMethodSession session,
+ in IMultiClientInputMethodSession multiClientSession, in InputChannel writeChannel);
+ void reportImeWindowTarget(int clientId, int targetWindowHandle, in IBinder imeWindowToken);
+ boolean isUidAllowedOnDisplay(int displayId, int uid);
+}
diff --git a/core/java/com/android/internal/inputmethod/IMultiClientInputMethodSession.aidl b/core/java/com/android/internal/inputmethod/IMultiClientInputMethodSession.aidl
new file mode 100644
index 0000000..b40e82c
--- /dev/null
+++ b/core/java/com/android/internal/inputmethod/IMultiClientInputMethodSession.aidl
@@ -0,0 +1,29 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.android.internal.inputmethod;
+
+import android.os.ResultReceiver;
+import android.view.inputmethod.EditorInfo;
+import com.android.internal.view.IInputContext;
+
+oneway interface IMultiClientInputMethodSession {
+ void startInputOrWindowGainedFocus(
+ in IInputContext inputContext, int missingMethods, in EditorInfo attribute,
+ int controlFlags, int softInputMode, int targetWindowHandle);
+ void showSoftInput(int flags, in ResultReceiver resultReceiver);
+ void hideSoftInput(int flags, in ResultReceiver resultReceiver);
+}
diff --git a/core/java/com/android/internal/inputmethod/MultiClientInputMethodPrivilegedOperations.java b/core/java/com/android/internal/inputmethod/MultiClientInputMethodPrivilegedOperations.java
new file mode 100644
index 0000000..9220117
--- /dev/null
+++ b/core/java/com/android/internal/inputmethod/MultiClientInputMethodPrivilegedOperations.java
@@ -0,0 +1,215 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.android.internal.inputmethod;
+
+import android.annotation.AnyThread;
+import android.annotation.Nullable;
+import android.os.IBinder;
+import android.os.RemoteException;
+import android.util.Log;
+import android.view.InputChannel;
+
+import com.android.internal.annotations.GuardedBy;
+import com.android.internal.view.IInputMethodSession;
+
+/**
+ * A utility class to take care of boilerplate code around IPCs.
+ *
+ * <p>Note: All public methods are designed to be thread-safe.</p>
+ */
+public class MultiClientInputMethodPrivilegedOperations {
+ private static final String TAG = "MultiClientInputMethodPrivilegedOperations";
+
+ private static final class OpsHolder {
+ @Nullable
+ @GuardedBy("this")
+ private IMultiClientInputMethodPrivilegedOperations mPrivOps;
+
+ /**
+ * Sets {@link IMultiClientInputMethodPrivilegedOperations}.
+ *
+ * <p>This method can be called only once.</p>
+ *
+ * @param privOps Binder interface to be set.
+ */
+ @AnyThread
+ public synchronized void set(IMultiClientInputMethodPrivilegedOperations privOps) {
+ if (mPrivOps != null) {
+ throw new IllegalStateException(
+ "IMultiClientInputMethodPrivilegedOperations must be set at most once."
+ + " privOps=" + privOps);
+ }
+ mPrivOps = privOps;
+ }
+
+ /**
+ * A simplified version of {@link android.os.Debug#getCaller()}.
+ *
+ * @return method name of the caller.
+ */
+ @AnyThread
+ private static String getCallerMethodName() {
+ final StackTraceElement[] callStack = Thread.currentThread().getStackTrace();
+ if (callStack.length <= 4) {
+ return "<bottom of call stack>";
+ }
+ return callStack[4].getMethodName();
+ }
+
+ @AnyThread
+ public synchronized void dispose() {
+ mPrivOps = null;
+ }
+
+ @AnyThread
+ @Nullable
+ public synchronized IMultiClientInputMethodPrivilegedOperations getAndWarnIfNull() {
+ if (mPrivOps == null) {
+ Log.e(TAG, getCallerMethodName() + " is ignored."
+ + " Call it within attachToken() and InputMethodService.onDestroy()");
+ }
+ return mPrivOps;
+ }
+ }
+ private final OpsHolder mOps = new OpsHolder();
+
+ /**
+ * Sets {@link IMultiClientInputMethodPrivilegedOperations}.
+ *
+ * <p>This method can be called only once.</p>
+ *
+ * @param privOps Binder interface to be set.
+ */
+ @AnyThread
+ public void set(IMultiClientInputMethodPrivilegedOperations privOps) {
+ mOps.set(privOps);
+ }
+
+ /**
+ * Disposes internal Binder proxy so that the real Binder object can be garbage collected.
+ */
+ @AnyThread
+ public void dispose() {
+ mOps.dispose();
+ }
+
+ /**
+
+ * Calls {@link IMultiClientInputMethodPrivilegedOperations#createInputMethodWindowToken(int)}.
+ *
+ * @param displayId display ID on which the IME window will be shown.
+ * @return Window token to be specified to the IME window.
+ */
+ @AnyThread
+ public IBinder createInputMethodWindowToken(int displayId) {
+ IMultiClientInputMethodPrivilegedOperations ops = mOps.getAndWarnIfNull();
+ if (ops == null) {
+ return null;
+ }
+ try {
+ return ops.createInputMethodWindowToken(displayId);
+ } catch (RemoteException e) {
+ throw e.rethrowFromSystemServer();
+ }
+ }
+
+ /**
+ * Calls {@link
+ * IMultiClientInputMethodPrivilegedOperations#deleteInputMethodWindowToken(IBinder)}.
+ *
+ * @param token Window token that is to be deleted.
+ */
+ @AnyThread
+ public void deleteInputMethodWindowToken(IBinder token) {
+ IMultiClientInputMethodPrivilegedOperations ops = mOps.getAndWarnIfNull();
+ if (ops == null) {
+ return;
+ }
+ try {
+ ops.deleteInputMethodWindowToken(token);
+ } catch (RemoteException e) {
+ throw e.rethrowFromSystemServer();
+ }
+ }
+
+ /**
+ * Calls {@link IMultiClientInputMethodPrivilegedOperations#acceptClient(int,
+ * IInputMethodSession, IMultiClientInputMethodSession, InputChannel)}.
+ *
+ * @param clientId client ID to be accepted.
+ * @param session {@link IInputMethodSession} that is also used for traditional IME protocol.
+ * @param multiClientSession {@link IMultiClientInputMethodSession} that defines new callbacks
+ * for multi-client scenarios.
+ * @param writeChannel {@link InputChannel} that is also used for traditional IME protocol.
+ */
+ @AnyThread
+ public void acceptClient(int clientId, IInputMethodSession session,
+ IMultiClientInputMethodSession multiClientSession, InputChannel writeChannel) {
+ final IMultiClientInputMethodPrivilegedOperations ops = mOps.getAndWarnIfNull();
+ if (ops == null) {
+ return;
+ }
+ try {
+ ops.acceptClient(clientId, session, multiClientSession, writeChannel);
+ } catch (RemoteException e) {
+ throw e.rethrowFromSystemServer();
+ }
+ }
+
+ /**
+ * Calls {@link IMultiClientInputMethodPrivilegedOperations#reportImeWindowTarget(int, int,
+ * IBinder)}.
+ *
+ * @param clientId client ID about which new IME target window is reported.
+ * @param targetWindowHandle integer handle of the target window.
+ * @param imeWindowToken {@link IBinder} window token of the IME window.
+ */
+ @AnyThread
+ public void reportImeWindowTarget(int clientId, int targetWindowHandle,
+ IBinder imeWindowToken) {
+ final IMultiClientInputMethodPrivilegedOperations ops = mOps.getAndWarnIfNull();
+ if (ops == null) {
+ return;
+ }
+ try {
+ ops.reportImeWindowTarget(clientId, targetWindowHandle, imeWindowToken);
+ } catch (RemoteException e) {
+ throw e.rethrowFromSystemServer();
+ }
+ }
+
+ /**
+ * Calls {@link IMultiClientInputMethodPrivilegedOperations#isUidAllowedOnDisplay(int, int)}.
+ *
+ * @param displayId display ID to be verified.
+ * @param uid UID to be verified.
+ * @return {@code true} when {@code uid} is allowed to access to {@code displayId}.
+ */
+ @AnyThread
+ public boolean isUidAllowedOnDisplay(int displayId, int uid) {
+ final IMultiClientInputMethodPrivilegedOperations ops = mOps.getAndWarnIfNull();
+ if (ops == null) {
+ return false;
+ }
+ try {
+ return ops.isUidAllowedOnDisplay(displayId, uid);
+ } catch (RemoteException e) {
+ throw e.rethrowFromSystemServer();
+ }
+ }
+
+}
diff --git a/core/java/com/android/internal/view/InputBindResult.java b/core/java/com/android/internal/view/InputBindResult.java
index ec8e8da..901cfe3 100644
--- a/core/java/com/android/internal/view/InputBindResult.java
+++ b/core/java/com/android/internal/view/InputBindResult.java
@@ -54,6 +54,7 @@
ResultCode.ERROR_NO_EDITOR,
ResultCode.ERROR_DISPLAY_ID_MISMATCH,
ResultCode.ERROR_INVALID_DISPLAY_ID,
+ ResultCode.ERROR_INVALID_CLIENT,
})
public @interface ResultCode {
/**
@@ -158,6 +159,10 @@
* display.
*/
int ERROR_INVALID_DISPLAY_ID = 14;
+ /**
+ * Indicates that the client is not recognized by the system.
+ */
+ int ERROR_INVALID_CLIENT = 15;
}
@ResultCode
@@ -287,6 +292,8 @@
return "ERROR_DISPLAY_ID_MISMATCH";
case ResultCode.ERROR_INVALID_DISPLAY_ID:
return "ERROR_INVALID_DISPLAY_ID";
+ case ResultCode.ERROR_INVALID_CLIENT:
+ return "ERROR_INVALID_CLIENT";
default:
return "Unknown(" + result + ")";
}
@@ -343,4 +350,9 @@
*/
public static final InputBindResult INVALID_DISPLAY_ID =
error(ResultCode.ERROR_INVALID_DISPLAY_ID);
+
+ /**
+ * Predefined error object for {@link ResultCode#ERROR_INVALID_CLIENT}.
+ */
+ public static final InputBindResult INVALID_CLIENT = error(ResultCode.ERROR_INVALID_CLIENT);
}
diff --git a/services/core/java/com/android/server/inputmethod/MultiClientInputMethodManagerService.java b/services/core/java/com/android/server/inputmethod/MultiClientInputMethodManagerService.java
new file mode 100644
index 0000000..2b67fe7
--- /dev/null
+++ b/services/core/java/com/android/server/inputmethod/MultiClientInputMethodManagerService.java
@@ -0,0 +1,1630 @@
+/*
+ * Copyright (C) 2018 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.android.server.inputmethod;
+
+import static android.view.WindowManager.LayoutParams.TYPE_INPUT_METHOD;
+
+import static java.lang.annotation.RetentionPolicy.SOURCE;
+
+import android.annotation.AnyThread;
+import android.annotation.BinderThread;
+import android.annotation.IntDef;
+import android.annotation.MainThread;
+import android.annotation.Nullable;
+import android.annotation.UserIdInt;
+import android.annotation.WorkerThread;
+import android.app.AppOpsManager;
+import android.app.PendingIntent;
+import android.content.BroadcastReceiver;
+import android.content.ComponentName;
+import android.content.Context;
+import android.content.Intent;
+import android.content.IntentFilter;
+import android.content.ServiceConnection;
+import android.content.pm.ApplicationInfo;
+import android.content.pm.PackageManager;
+import android.content.pm.ResolveInfo;
+import android.content.pm.ServiceInfo;
+import android.inputmethodservice.MultiClientInputMethodServiceDelegate;
+import android.net.Uri;
+import android.os.Binder;
+import android.os.Build;
+import android.os.Debug;
+import android.os.Handler;
+import android.os.HandlerThread;
+import android.os.IBinder;
+import android.os.RemoteException;
+import android.os.ResultReceiver;
+import android.os.ShellCallback;
+import android.os.SystemProperties;
+import android.os.UserHandle;
+import android.provider.Settings;
+import android.text.TextUtils;
+import android.text.style.SuggestionSpan;
+import android.util.ArrayMap;
+import android.util.ArraySet;
+import android.util.Slog;
+import android.util.SparseArray;
+import android.view.InputChannel;
+import android.view.WindowManager.LayoutParams.SoftInputModeFlags;
+import android.view.inputmethod.EditorInfo;
+import android.view.inputmethod.InputConnectionInspector.MissingMethodFlags;
+import android.view.inputmethod.InputMethodInfo;
+import android.view.inputmethod.InputMethodSubtype;
+
+import com.android.internal.annotations.GuardedBy;
+import com.android.internal.inputmethod.IMultiClientInputMethod;
+import com.android.internal.inputmethod.IMultiClientInputMethodPrivilegedOperations;
+import com.android.internal.inputmethod.IMultiClientInputMethodSession;
+import com.android.internal.inputmethod.StartInputFlags;
+import com.android.internal.inputmethod.StartInputReason;
+import com.android.internal.inputmethod.UnbindReason;
+import com.android.internal.util.function.pooled.PooledLambda;
+import com.android.internal.view.IInputContext;
+import com.android.internal.view.IInputMethodClient;
+import com.android.internal.view.IInputMethodManager;
+import com.android.internal.view.IInputMethodSession;
+import com.android.internal.view.InputBindResult;
+import com.android.server.LocalServices;
+import com.android.server.SystemService;
+import com.android.server.wm.WindowManagerInternal;
+
+import java.io.FileDescriptor;
+import java.lang.annotation.Retention;
+import java.util.Collections;
+import java.util.List;
+import java.util.WeakHashMap;
+
+/**
+ * Actual implementation of multi-client InputMethodManagerService.
+ *
+ * <p>This system service is intentionally compatible with {@link InputMethodManagerService} so that
+ * we can switch the implementation at the boot time.</p>
+ */
+public final class MultiClientInputMethodManagerService {
+ static final String TAG = "MultiClientInputMethodManagerService";
+ static final boolean DEBUG = false;
+
+ /**
+ * System property key for the production use. The value must be either empty or a valid
+ * (flattened) component name of the multi-client IME.
+ */
+ private static final String PROP_PROD_MULTI_CLIENT_IME = "ro.sys.multi_client_ime";
+
+ /**
+ * System property key for debugging purpose. The value must be either empty or a valid
+ * (flattened) component name of the multi-client IME.
+ *
+ * <p>This value will be ignored when {@link Build#IS_DEBUGGABLE} returns {@code false}</p>
+ */
+ private static final String PROP_DEBUG_MULTI_CLIENT_IME = "persist.debug.multi_client_ime";
+
+ private static final long RECONNECT_DELAY_MSEC = 1000;
+
+ /**
+ * Unlike {@link InputMethodManagerService}, {@link MultiClientInputMethodManagerService}
+ * always binds to the IME with {@link Context#BIND_FOREGROUND_SERVICE} for now for simplicity.
+ */
+ private static final int IME_CONNECTION_UNIFIED_BIND_FLAGS =
+ Context.BIND_AUTO_CREATE
+ | Context.BIND_NOT_VISIBLE
+ | Context.BIND_NOT_FOREGROUND
+ | Context.BIND_FOREGROUND_SERVICE;
+
+ /**
+ * Inner class to read system property on demand, not when
+ * {@link MultiClientInputMethodManagerService} class is accessed.
+ */
+ private static final class ImeComponentName {
+ private static ComponentName evaluate() {
+ if (Build.IS_DEBUGGABLE) {
+ // If debuggable, allow developers to override the multi-client IME component name
+ // with a different (writable) key.
+ final ComponentName debugIme = ComponentName.unflattenFromString(
+ SystemProperties.get(PROP_DEBUG_MULTI_CLIENT_IME, ""));
+ if (debugIme != null) {
+ return debugIme;
+ }
+ }
+ return ComponentName.unflattenFromString(
+ SystemProperties.get(PROP_PROD_MULTI_CLIENT_IME, ""));
+ }
+
+ /**
+ * {@link ComponentName} of the multi-client IME. {@code null} when the system is not
+ * configured to use multi-client IME.
+ */
+ @Nullable
+ static final ComponentName sValue = evaluate();
+ }
+
+ public static boolean isConfiguredToUse() {
+ return ImeComponentName.sValue != null;
+ }
+
+ private static void reportNotSupported() {
+ if (DEBUG) {
+ Slog.d(TAG, "non-supported operation. callers=" + Debug.getCallers(3));
+ }
+ }
+
+ /**
+ * {@link MultiClientInputMethodManagerService} is not intended to be instantiated.
+ */
+ private MultiClientInputMethodManagerService() {
+ }
+
+ /**
+ * The implementation of {@link SystemService} for multi-client IME.
+ */
+ public static final class Lifecycle extends SystemService {
+ private final ApiCallbacks mApiCallbacks;
+ private final OnWorkerThreadCallback mOnWorkerThreadCallback;
+
+ @MainThread
+ public Lifecycle(Context context) {
+ super(context);
+
+ final UserToInputMethodInfoMap userIdToInputMethodInfoMapper =
+ new UserToInputMethodInfoMap();
+ final UserDataMap userDataMap = new UserDataMap();
+ final HandlerThread workerThread = new HandlerThread(TAG);
+ workerThread.start();
+ mApiCallbacks = new ApiCallbacks(context, userDataMap, userIdToInputMethodInfoMapper);
+ mOnWorkerThreadCallback = new OnWorkerThreadCallback(
+ context, userDataMap, userIdToInputMethodInfoMapper,
+ new Handler(workerThread.getLooper(), msg -> false, true));
+
+ LocalServices.addService(InputMethodManagerInternal.class,
+ new InputMethodManagerInternal() {
+ @Override
+ public void setInteractive(boolean interactive) {
+ reportNotSupported();
+ }
+
+ @Override
+ public void hideCurrentInputMethod() {
+ reportNotSupported();
+ }
+
+ @Override
+ public void startVrInputMethodNoCheck(ComponentName componentName) {
+ reportNotSupported();
+ }
+ });
+ }
+
+ @MainThread
+ @Override
+ public void onBootPhase(int phase) {
+ mOnWorkerThreadCallback.getHandler().sendMessage(PooledLambda.obtainMessage(
+ OnWorkerThreadCallback::onBootPhase, mOnWorkerThreadCallback, phase));
+ }
+
+ @MainThread
+ @Override
+ public void onStart() {
+ publishBinderService(Context.INPUT_METHOD_SERVICE, mApiCallbacks);
+ }
+
+ @MainThread
+ @Override
+ public void onStartUser(@UserIdInt int userId) {
+ mOnWorkerThreadCallback.getHandler().sendMessage(PooledLambda.obtainMessage(
+ OnWorkerThreadCallback::onStartUser, mOnWorkerThreadCallback, userId));
+ }
+
+ @MainThread
+ @Override
+ public void onUnlockUser(@UserIdInt int userId) {
+ mOnWorkerThreadCallback.getHandler().sendMessage(PooledLambda.obtainMessage(
+ OnWorkerThreadCallback::onUnlockUser, mOnWorkerThreadCallback, userId));
+ }
+
+ @MainThread
+ @Override
+ public void onStopUser(@UserIdInt int userId) {
+ mOnWorkerThreadCallback.getHandler().sendMessage(PooledLambda.obtainMessage(
+ OnWorkerThreadCallback::onStopUser, mOnWorkerThreadCallback, userId));
+ }
+ }
+
+ private static final class OnWorkerThreadCallback {
+ private final Context mContext;
+ private final UserDataMap mUserDataMap;
+ private final UserToInputMethodInfoMap mInputMethodInfoMap;
+ private final Handler mHandler;
+
+ OnWorkerThreadCallback(Context context, UserDataMap userDataMap,
+ UserToInputMethodInfoMap inputMethodInfoMap, Handler handler) {
+ mContext = context;
+ mUserDataMap = userDataMap;
+ mInputMethodInfoMap = inputMethodInfoMap;
+ mHandler = handler;
+ }
+
+ @AnyThread
+ Handler getHandler() {
+ return mHandler;
+ }
+
+ @WorkerThread
+ private void tryBindInputMethodService(@UserIdInt int userId) {
+ final PerUserData data = mUserDataMap.get(userId);
+ if (data == null) {
+ Slog.i(TAG, "tryBindInputMethodService is called for an unknown user=" + userId);
+ return;
+ }
+
+ final InputMethodInfo imi = queryInputMethod(mContext, userId, ImeComponentName.sValue);
+ if (imi == null) {
+ Slog.w(TAG, "Multi-client InputMethod is not found. component="
+ + ImeComponentName.sValue);
+ synchronized (data.mLock) {
+ switch (data.mState) {
+ case PerUserState.USER_LOCKED:
+ case PerUserState.SERVICE_NOT_QUERIED:
+ case PerUserState.SERVICE_RECOGNIZED:
+ case PerUserState.UNBIND_CALLED:
+ // Safe to clean up.
+ mInputMethodInfoMap.remove(userId);
+ break;
+ }
+ }
+ return;
+ }
+
+ synchronized (data.mLock) {
+ switch (data.mState) {
+ case PerUserState.USER_LOCKED:
+ // If the user is still locked, we currently do not try to start IME.
+ return;
+ case PerUserState.SERVICE_NOT_QUERIED:
+ case PerUserState.SERVICE_RECOGNIZED:
+ case PerUserState.UNBIND_CALLED:
+ break;
+ case PerUserState.WAITING_SERVICE_CONNECTED:
+ case PerUserState.SERVICE_CONNECTED:
+ // OK, nothing to do.
+ return;
+ default:
+ Slog.wtf(TAG, "Unknown state=" + data.mState);
+ return;
+ }
+ data.mState = PerUserState.SERVICE_RECOGNIZED;
+ data.mCurrentInputMethodInfo = imi;
+ mInputMethodInfoMap.put(userId, imi);
+ final boolean bindResult = data.bindServiceLocked(mContext, userId);
+ if (!bindResult) {
+ Slog.e(TAG, "Failed to bind Multi-client InputMethod.");
+ return;
+ }
+ data.mState = PerUserState.WAITING_SERVICE_CONNECTED;
+ }
+ }
+
+ @WorkerThread
+ void onStartUser(@UserIdInt int userId) {
+ if (DEBUG) {
+ Slog.v(TAG, "onStartUser userId=" + userId);
+ }
+ final PerUserData data = new PerUserData(userId, null, PerUserState.USER_LOCKED, this);
+ mUserDataMap.put(userId, data);
+ }
+
+ @WorkerThread
+ void onUnlockUser(@UserIdInt int userId) {
+ if (DEBUG) {
+ Slog.v(TAG, "onUnlockUser() userId=" + userId);
+ }
+ final PerUserData data = mUserDataMap.get(userId);
+ if (data == null) {
+ Slog.i(TAG, "onUnlockUser is called for an unknown user=" + userId);
+ return;
+ }
+ synchronized (data.mLock) {
+ switch (data.mState) {
+ case PerUserState.USER_LOCKED:
+ data.mState = PerUserState.SERVICE_NOT_QUERIED;
+ tryBindInputMethodService(userId);
+ break;
+ default:
+ Slog.wtf(TAG, "Unknown state=" + data.mState);
+ break;
+ }
+ }
+ }
+
+ @WorkerThread
+ void onStopUser(@UserIdInt int userId) {
+ if (DEBUG) {
+ Slog.v(TAG, "onStopUser() userId=" + userId);
+ }
+ mInputMethodInfoMap.remove(userId);
+ final PerUserData data = mUserDataMap.removeReturnOld(userId);
+ if (data == null) {
+ Slog.i(TAG, "onStopUser is called for an unknown user=" + userId);
+ return;
+ }
+ synchronized (data.mLock) {
+ switch (data.mState) {
+ case PerUserState.USER_LOCKED:
+ case PerUserState.SERVICE_RECOGNIZED:
+ case PerUserState.UNBIND_CALLED:
+ // OK, nothing to do.
+ return;
+ case PerUserState.SERVICE_CONNECTED:
+ case PerUserState.WAITING_SERVICE_CONNECTED:
+ break;
+ default:
+ Slog.wtf(TAG, "Unknown state=" + data.mState);
+ break;
+ }
+ data.unbindServiceLocked(mContext);
+ data.mState = PerUserState.UNBIND_CALLED;
+ data.mCurrentInputMethod = null;
+
+ // When a Service is explicitly unbound with Context.unbindService(),
+ // onServiceDisconnected() will not be triggered. Hence here we explicitly call
+ // onInputMethodDisconnectedLocked() as if the Service is already gone.
+ data.onInputMethodDisconnectedLocked();
+ }
+ }
+
+ @WorkerThread
+ void onServiceConnected(PerUserData data, IMultiClientInputMethod service) {
+ if (DEBUG) {
+ Slog.v(TAG, "onServiceConnected() data.mUserId=" + data.mUserId);
+ }
+ synchronized (data.mLock) {
+ switch (data.mState) {
+ case PerUserState.UNBIND_CALLED:
+ // We should ignore this callback.
+ return;
+ case PerUserState.WAITING_SERVICE_CONNECTED:
+ // OK.
+ data.mState = PerUserState.SERVICE_CONNECTED;
+ data.mCurrentInputMethod = service;
+ try {
+ data.mCurrentInputMethod.initialize(new ImeCallbacks(data));
+ } catch (RemoteException e) {
+ }
+ data.onInputMethodConnectedLocked();
+ break;
+ default:
+ Slog.wtf(TAG, "Unknown state=" + data.mState);
+ return;
+ }
+ }
+ }
+
+ @WorkerThread
+ void onServiceDisconnected(PerUserData data) {
+ if (DEBUG) {
+ Slog.v(TAG, "onServiceDisconnected() data.mUserId=" + data.mUserId);
+ }
+ final WindowManagerInternal windowManagerInternal =
+ LocalServices.getService(WindowManagerInternal.class);
+ synchronized (data.mLock) {
+ // We assume the number of tokens would not be that large (up to 10 or so) hence
+ // linear search should be acceptable.
+ final int numTokens = data.mDisplayIdToImeWindowTokenMap.size();
+ for (int i = 0; i < numTokens; ++i) {
+ final TokenInfo info = data.mDisplayIdToImeWindowTokenMap.valueAt(i);
+ windowManagerInternal.removeWindowToken(info.mToken, false, info.mDisplayId);
+ }
+ data.mDisplayIdToImeWindowTokenMap.clear();
+ switch (data.mState) {
+ case PerUserState.UNBIND_CALLED:
+ // We should ignore this callback.
+ return;
+ case PerUserState.WAITING_SERVICE_CONNECTED:
+ case PerUserState.SERVICE_CONNECTED:
+ // onServiceDisconnected() means the biding is still alive.
+ data.mState = PerUserState.WAITING_SERVICE_CONNECTED;
+ data.mCurrentInputMethod = null;
+ data.onInputMethodDisconnectedLocked();
+ break;
+ default:
+ Slog.wtf(TAG, "Unknown state=" + data.mState);
+ return;
+ }
+ }
+ }
+
+ @WorkerThread
+ void onBindingDied(PerUserData data) {
+ if (DEBUG) {
+ Slog.v(TAG, "onBindingDied() data.mUserId=" + data.mUserId);
+ }
+ final WindowManagerInternal windowManagerInternal =
+ LocalServices.getService(WindowManagerInternal.class);
+ synchronized (data.mLock) {
+ // We assume the number of tokens would not be that large (up to 10 or so) hence
+ // linear search should be acceptable.
+ final int numTokens = data.mDisplayIdToImeWindowTokenMap.size();
+ for (int i = 0; i < numTokens; ++i) {
+ final TokenInfo info = data.mDisplayIdToImeWindowTokenMap.valueAt(i);
+ windowManagerInternal.removeWindowToken(info.mToken, false, info.mDisplayId);
+ }
+ data.mDisplayIdToImeWindowTokenMap.clear();
+ switch (data.mState) {
+ case PerUserState.UNBIND_CALLED:
+ // We should ignore this callback.
+ return;
+ case PerUserState.WAITING_SERVICE_CONNECTED:
+ case PerUserState.SERVICE_CONNECTED: {
+ // onBindingDied() means the biding is dead.
+ data.mState = PerUserState.UNBIND_CALLED;
+ data.mCurrentInputMethod = null;
+ data.onInputMethodDisconnectedLocked();
+ // Schedule a retry
+ mHandler.sendMessageDelayed(PooledLambda.obtainMessage(
+ OnWorkerThreadCallback::tryBindInputMethodService,
+ this, data.mUserId), RECONNECT_DELAY_MSEC);
+ break;
+ }
+ default:
+ Slog.wtf(TAG, "Unknown state=" + data.mState);
+ return;
+ }
+ }
+ }
+
+ @WorkerThread
+ void onBootPhase(int phase) {
+ if (DEBUG) {
+ Slog.v(TAG, "onBootPhase() phase=" + phase);
+ }
+ switch (phase) {
+ case SystemService.PHASE_ACTIVITY_MANAGER_READY: {
+ final IntentFilter filter = new IntentFilter(Intent.ACTION_PACKAGE_ADDED);
+ filter.addDataScheme("package");
+ mContext.registerReceiver(new BroadcastReceiver() {
+ @Override
+ public void onReceive(Context context, Intent intent) {
+ onPackageAdded(intent);
+ }
+ }, filter, null, mHandler);
+ }
+ break;
+ }
+ }
+
+ @WorkerThread
+ void onPackageAdded(Intent intent) {
+ if (DEBUG) {
+ Slog.v(TAG, "onPackageAdded() intent=" + intent);
+ }
+ final Uri uri = intent.getData();
+ if (uri == null) {
+ return;
+ }
+ if (!intent.hasExtra(Intent.EXTRA_UID)) {
+ return;
+ }
+ final String packageName = uri.getSchemeSpecificPart();
+ if (ImeComponentName.sValue == null
+ || packageName == null
+ || !TextUtils.equals(ImeComponentName.sValue.getPackageName(), packageName)) {
+ return;
+ }
+ final int userId = UserHandle.getUserId(intent.getIntExtra(Intent.EXTRA_UID, 0));
+ tryBindInputMethodService(userId);
+ }
+ }
+
+ private static final class WindowInfo {
+ final IBinder mWindowToken;
+ final int mWindowHandle;
+
+ WindowInfo(IBinder windowToken, int windowCookie) {
+ mWindowToken = windowToken;
+ mWindowHandle = windowCookie;
+ }
+ }
+
+ /**
+ * Describes the state of each IME client.
+ */
+ @Retention(SOURCE)
+ @IntDef({InputMethodClientState.REGISTERED,
+ InputMethodClientState.WAITING_FOR_IME_SESSION,
+ InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT,
+ InputMethodClientState.ALREADY_SENT_BIND_RESULT,
+ InputMethodClientState.UNREGISTERED})
+ private @interface InputMethodClientState {
+ /**
+ * {@link IInputMethodManager#addClient(IInputMethodClient, IInputContext, int)} is called
+ * and this client is now recognized by the system. When the system lost the connection to
+ * the current IME, all the clients need to be re-initialized from this state.
+ */
+ int REGISTERED = 1;
+ /**
+ * This client is notified to the current IME with {@link
+ * IMultiClientInputMethod#addClient(int, int, int, int)} but the IME is not yet responded
+ * with {@link IMultiClientInputMethodPrivilegedOperations#acceptClient(int,
+ * IInputMethodSession, IMultiClientInputMethodSession, InputChannel)}.
+ */
+ int WAITING_FOR_IME_SESSION = 2;
+ /**
+ * This client is already accepted by the IME but a valid {@link InputBindResult} has not
+ * been returned to the client yet.
+ */
+ int READY_TO_SEND_FIRST_BIND_RESULT = 3;
+ /**
+ * This client has already received a valid {@link InputBindResult} at least once. This
+ * means that the client can directly call {@link IInputMethodSession} IPCs and key events
+ * via {@link InputChannel}. When the current IME is unbound, these client end points also
+ * need to be cleared.
+ */
+ int ALREADY_SENT_BIND_RESULT = 4;
+ /**
+ * The client process is dying.
+ */
+ int UNREGISTERED = 5;
+ }
+
+ private static final class InputMethodClientIdSource {
+ @GuardedBy("InputMethodClientIdSource.class")
+ private static int sNextValue = 0;
+
+ private InputMethodClientIdSource() {
+ }
+
+ static synchronized int getNext() {
+ final int result = sNextValue;
+ sNextValue++;
+ if (sNextValue < 0) {
+ sNextValue = 0;
+ }
+ return result;
+ }
+ }
+
+ private static final class WindowHandleSource {
+ @GuardedBy("WindowHandleSource.class")
+ private static int sNextValue = 0;
+
+ private WindowHandleSource() {
+ }
+
+ static synchronized int getNext() {
+ final int result = sNextValue;
+ sNextValue++;
+ if (sNextValue < 0) {
+ sNextValue = 0;
+ }
+ return result;
+ }
+ }
+
+ private static final class InputMethodClientInfo {
+ final IInputMethodClient mClient;
+ final int mUid;
+ final int mPid;
+ final int mSelfReportedDisplayId;
+ final int mClientId;
+
+ @GuardedBy("PerUserData.mLock")
+ @InputMethodClientState
+ int mState;
+ @GuardedBy("PerUserData.mLock")
+ int mBindingSequence;
+ @GuardedBy("PerUserData.mLock")
+ InputChannel mWriteChannel;
+ @GuardedBy("PerUserData.mLock")
+ IInputMethodSession mInputMethodSession;
+ @GuardedBy("PerUserData.mLock")
+ IMultiClientInputMethodSession mMSInputMethodSession;
+ @GuardedBy("PerUserData.mLock")
+ final WeakHashMap<IBinder, WindowInfo> mWindowMap = new WeakHashMap<>();
+
+ InputMethodClientInfo(IInputMethodClient client, int uid, int pid,
+ int selfReportedDisplayId) {
+ mClient = client;
+ mUid = uid;
+ mPid = pid;
+ mSelfReportedDisplayId = selfReportedDisplayId;
+ mClientId = InputMethodClientIdSource.getNext();
+ }
+ }
+
+ private static final class UserDataMap {
+ @GuardedBy("mMap")
+ private final SparseArray<PerUserData> mMap = new SparseArray<>();
+
+ @AnyThread
+ @Nullable
+ PerUserData get(@UserIdInt int userId) {
+ synchronized (mMap) {
+ return mMap.get(userId);
+ }
+ }
+
+ @AnyThread
+ void put(@UserIdInt int userId, PerUserData data) {
+ synchronized (mMap) {
+ mMap.put(userId, data);
+ }
+ }
+
+ @AnyThread
+ @Nullable
+ PerUserData removeReturnOld(@UserIdInt int userId) {
+ synchronized (mMap) {
+ return mMap.removeReturnOld(userId);
+ }
+ }
+ }
+
+ private static final class TokenInfo {
+ final Binder mToken;
+ final int mDisplayId;
+ TokenInfo(Binder token, int displayId) {
+ mToken = token;
+ mDisplayId = displayId;
+ }
+ }
+
+ @Retention(SOURCE)
+ @IntDef({
+ PerUserState.USER_LOCKED,
+ PerUserState.SERVICE_NOT_QUERIED,
+ PerUserState.SERVICE_RECOGNIZED,
+ PerUserState.WAITING_SERVICE_CONNECTED,
+ PerUserState.SERVICE_CONNECTED,
+ PerUserState.UNBIND_CALLED})
+ private @interface PerUserState {
+ /**
+ * The user is still locked.
+ */
+ int USER_LOCKED = 1;
+ /**
+ * The system has not queried whether there is a multi-client IME or not.
+ */
+ int SERVICE_NOT_QUERIED = 2;
+ /**
+ * A multi-client IME specified in {@link #PROP_DEBUG_MULTI_CLIENT_IME} is found in the
+ * system, but not bound yet.
+ */
+ int SERVICE_RECOGNIZED = 3;
+ /**
+ * {@link Context#bindServiceAsUser(Intent, ServiceConnection, int, Handler, UserHandle)} is
+ * already called for the IME but
+ * {@link ServiceConnection#onServiceConnected(ComponentName, IBinder)} is not yet called
+ * back. This includes once the IME is bound but temporarily disconnected as notified with
+ * {@link ServiceConnection#onServiceDisconnected(ComponentName)}.
+ */
+ int WAITING_SERVICE_CONNECTED = 4;
+ /**
+ * {@link ServiceConnection#onServiceConnected(ComponentName, IBinder)} is already called
+ * back. The IME is ready to be used.
+ */
+ int SERVICE_CONNECTED = 5;
+ /**
+ * The binding is gone. Either {@link Context#unbindService(ServiceConnection)} is
+ * explicitly called or the system decided to destroy the binding as notified with
+ * {@link ServiceConnection#onBindingDied(ComponentName)}.
+ */
+ int UNBIND_CALLED = 6;
+ }
+
+ /**
+ * Takes care of per-user state separation.
+ */
+ private static final class PerUserData {
+ final Object mLock = new Object();
+
+ /**
+ * User ID (not UID) that is associated with this data.
+ */
+ @UserIdInt
+ private final int mUserId;
+
+ /**
+ * {@link IMultiClientInputMethod} of the currently connected multi-client IME. This
+ * must be non-{@code null} only while {@link #mState} is
+ * {@link PerUserState#SERVICE_CONNECTED}.
+ */
+ @Nullable
+ @GuardedBy("mLock")
+ IMultiClientInputMethod mCurrentInputMethod;
+
+ /**
+ * {@link InputMethodInfo} of the currently selected multi-client IME. This must be
+ * non-{@code null} unless {@link #mState} is {@link PerUserState#SERVICE_NOT_QUERIED}.
+ */
+ @GuardedBy("mLock")
+ @Nullable
+ InputMethodInfo mCurrentInputMethodInfo;
+
+ /**
+ * Describes the current service state.
+ */
+ @GuardedBy("mLock")
+ @PerUserState
+ int mState;
+
+ /**
+ * A {@link SparseArray} that maps display ID to IME Window token that is already issued to
+ * the IME.
+ */
+ @GuardedBy("mLock")
+ final ArraySet<TokenInfo> mDisplayIdToImeWindowTokenMap = new ArraySet<>();
+
+ @GuardedBy("mLock")
+ private final ArrayMap<IBinder, InputMethodClientInfo> mClientMap = new ArrayMap<>();
+
+ @GuardedBy("mLock")
+ private SparseArray<InputMethodClientInfo> mClientIdToClientMap = new SparseArray<>();
+
+ private final OnWorkerThreadServiceConnection mOnWorkerThreadServiceConnection;
+
+ /**
+ * A {@link ServiceConnection} that is designed to run on a certain worker thread with
+ * which {@link OnWorkerThreadCallback} is associated.
+ *
+ * @see Context#bindServiceAsUser(Intent, ServiceConnection, int, Handler, UserHandle).
+ */
+ private static final class OnWorkerThreadServiceConnection implements ServiceConnection {
+ private final PerUserData mData;
+ private final OnWorkerThreadCallback mCallback;
+
+ OnWorkerThreadServiceConnection(PerUserData data, OnWorkerThreadCallback callback) {
+ mData = data;
+ mCallback = callback;
+ }
+
+ @WorkerThread
+ @Override
+ public void onServiceConnected(ComponentName name, IBinder service) {
+ mCallback.onServiceConnected(mData,
+ IMultiClientInputMethod.Stub.asInterface(service));
+ }
+
+ @WorkerThread
+ @Override
+ public void onServiceDisconnected(ComponentName name) {
+ mCallback.onServiceDisconnected(mData);
+ }
+
+ @WorkerThread
+ @Override
+ public void onBindingDied(ComponentName name) {
+ mCallback.onBindingDied(mData);
+ }
+
+ Handler getHandler() {
+ return mCallback.getHandler();
+ }
+ }
+
+ PerUserData(@UserIdInt int userId, @Nullable InputMethodInfo inputMethodInfo,
+ @PerUserState int initialState, OnWorkerThreadCallback callback) {
+ mUserId = userId;
+ mCurrentInputMethodInfo = inputMethodInfo;
+ mState = initialState;
+ mOnWorkerThreadServiceConnection =
+ new OnWorkerThreadServiceConnection(this, callback);
+ }
+
+ @GuardedBy("mLock")
+ boolean bindServiceLocked(Context context, @UserIdInt int userId) {
+ final Intent intent =
+ new Intent(MultiClientInputMethodServiceDelegate.SERVICE_INTERFACE)
+ .setComponent(mCurrentInputMethodInfo.getComponent())
+ .putExtra(Intent.EXTRA_CLIENT_LABEL,
+ com.android.internal.R.string.input_method_binding_label)
+ .putExtra(Intent.EXTRA_CLIENT_INTENT, PendingIntent.getActivity(
+ context, 0,
+ new Intent(Settings.ACTION_INPUT_METHOD_SETTINGS), 0));
+
+ // Note: Instead of re-dispatching callback from the main thread to the worker thread
+ // where OnWorkerThreadCallback is running, we pass the Handler object here so that
+ // the callbacks will be directly dispatched to the worker thread.
+ return context.bindServiceAsUser(intent, mOnWorkerThreadServiceConnection,
+ IME_CONNECTION_UNIFIED_BIND_FLAGS,
+ mOnWorkerThreadServiceConnection.getHandler(), UserHandle.of(userId));
+ }
+
+ @GuardedBy("mLock")
+ void unbindServiceLocked(Context context) {
+ context.unbindService(mOnWorkerThreadServiceConnection);
+ }
+
+ @GuardedBy("mLock")
+ @Nullable
+ InputMethodClientInfo getClientLocked(IInputMethodClient client) {
+ return mClientMap.get(client.asBinder());
+ }
+
+ @GuardedBy("mLock")
+ @Nullable
+ InputMethodClientInfo getClientFromIdLocked(int clientId) {
+ return mClientIdToClientMap.get(clientId);
+ }
+
+ @GuardedBy("mLock")
+ @Nullable
+ InputMethodClientInfo removeClientLocked(IInputMethodClient client) {
+ final InputMethodClientInfo info = mClientMap.remove(client.asBinder());
+ if (info != null) {
+ mClientIdToClientMap.remove(info.mClientId);
+ }
+ return info;
+ }
+
+ @GuardedBy("mLock")
+ void addClientLocked(int uid, int pid, IInputMethodClient client,
+ int selfReportedDisplayId) {
+ if (getClientLocked(client) != null) {
+ Slog.wtf(TAG, "The same client is added multiple times");
+ return;
+ }
+ final ClientDeathRecipient deathRecipient = new ClientDeathRecipient(this, client);
+ try {
+ client.asBinder().linkToDeath(deathRecipient, 0);
+ } catch (RemoteException e) {
+ throw new IllegalStateException(e);
+ }
+ final InputMethodClientInfo clientInfo =
+ new InputMethodClientInfo(client, uid, pid, selfReportedDisplayId);
+ clientInfo.mState = InputMethodClientState.REGISTERED;
+ mClientMap.put(client.asBinder(), clientInfo);
+ mClientIdToClientMap.put(clientInfo.mClientId, clientInfo);
+ switch (mState) {
+ case PerUserState.SERVICE_CONNECTED:
+ try {
+ mCurrentInputMethod.addClient(
+ clientInfo.mClientId, clientInfo.mPid, clientInfo.mUid,
+ clientInfo.mSelfReportedDisplayId);
+ clientInfo.mState = InputMethodClientState.WAITING_FOR_IME_SESSION;
+ } catch (RemoteException e) {
+ // TODO(yukawa): Need logging and expected behavior
+ }
+ break;
+ }
+ }
+
+ @GuardedBy("mLock")
+ void onInputMethodConnectedLocked() {
+ final int numClients = mClientMap.size();
+ for (int i = 0; i < numClients; ++i) {
+ final InputMethodClientInfo clientInfo = mClientMap.valueAt(i);
+ switch (clientInfo.mState) {
+ case InputMethodClientState.REGISTERED:
+ // OK
+ break;
+ default:
+ Slog.e(TAG, "Unexpected state=" + clientInfo.mState);
+ return;
+ }
+ try {
+ mCurrentInputMethod.addClient(
+ clientInfo.mClientId, clientInfo.mUid, clientInfo.mPid,
+ clientInfo.mSelfReportedDisplayId);
+ clientInfo.mState = InputMethodClientState.WAITING_FOR_IME_SESSION;
+ } catch (RemoteException e) {
+ }
+ }
+ }
+
+ @GuardedBy("mLock")
+ void onInputMethodDisconnectedLocked() {
+ final int numClients = mClientMap.size();
+ for (int i = 0; i < numClients; ++i) {
+ final InputMethodClientInfo clientInfo = mClientMap.valueAt(i);
+ switch (clientInfo.mState) {
+ case InputMethodClientState.REGISTERED:
+ // Disconnected before onInputMethodConnectedLocked().
+ break;
+ case InputMethodClientState.WAITING_FOR_IME_SESSION:
+ // Disconnected between addClient() and acceptClient().
+ clientInfo.mState = InputMethodClientState.REGISTERED;
+ break;
+ case InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT:
+ clientInfo.mState = InputMethodClientState.REGISTERED;
+ clientInfo.mInputMethodSession = null;
+ clientInfo.mMSInputMethodSession = null;
+ if (clientInfo.mWriteChannel != null) {
+ clientInfo.mWriteChannel.dispose();
+ clientInfo.mWriteChannel = null;
+ }
+ break;
+ case InputMethodClientState.ALREADY_SENT_BIND_RESULT:
+ try {
+ clientInfo.mClient.onUnbindMethod(clientInfo.mBindingSequence,
+ UnbindReason.DISCONNECT_IME);
+ } catch (RemoteException e) {
+ }
+ clientInfo.mState = InputMethodClientState.REGISTERED;
+ clientInfo.mInputMethodSession = null;
+ clientInfo.mMSInputMethodSession = null;
+ if (clientInfo.mWriteChannel != null) {
+ clientInfo.mWriteChannel.dispose();
+ clientInfo.mWriteChannel = null;
+ }
+ break;
+ }
+ }
+ }
+
+ private static final class ClientDeathRecipient implements IBinder.DeathRecipient {
+ private final PerUserData mPerUserData;
+ private final IInputMethodClient mClient;
+
+ ClientDeathRecipient(PerUserData perUserData, IInputMethodClient client) {
+ mPerUserData = perUserData;
+ mClient = client;
+ }
+
+ @BinderThread
+ @Override
+ public void binderDied() {
+ synchronized (mPerUserData.mLock) {
+ mClient.asBinder().unlinkToDeath(this, 0);
+
+ final InputMethodClientInfo clientInfo =
+ mPerUserData.removeClientLocked(mClient);
+ if (clientInfo == null) {
+ return;
+ }
+
+ if (clientInfo.mWriteChannel != null) {
+ clientInfo.mWriteChannel.dispose();
+ clientInfo.mWriteChannel = null;
+ }
+ if (clientInfo.mInputMethodSession != null) {
+ try {
+ clientInfo.mInputMethodSession.finishSession();
+ } catch (RemoteException e) {
+ }
+ clientInfo.mInputMethodSession = null;
+ }
+ clientInfo.mMSInputMethodSession = null;
+ clientInfo.mState = InputMethodClientState.UNREGISTERED;
+ switch (mPerUserData.mState) {
+ case PerUserState.SERVICE_CONNECTED:
+ try {
+ mPerUserData.mCurrentInputMethod.removeClient(clientInfo.mClientId);
+ } catch (RemoteException e) {
+ // TODO(yukawa): Need logging and expected behavior
+ }
+ break;
+ }
+ }
+ }
+ }
+ }
+
+ /**
+ * Queries for multi-client IME specified with {@code componentName}.
+ *
+ * @param context {@link Context} to be used to query component.
+ * @param userId User ID for which the multi-client IME is queried.
+ * @param componentName {@link ComponentName} to be queried.
+ * @return {@link InputMethodInfo} when multi-client IME is found. Otherwise {@code null}.
+ */
+ @Nullable
+ private static InputMethodInfo queryInputMethod(Context context, @UserIdInt int userId,
+ @Nullable ComponentName componentName) {
+ if (componentName == null) {
+ return null;
+ }
+
+ // Use for queryIntentServicesAsUser
+ final PackageManager pm = context.getPackageManager();
+ final List<ResolveInfo> services = pm.queryIntentServicesAsUser(
+ new Intent(MultiClientInputMethodServiceDelegate.SERVICE_INTERFACE)
+ .setComponent(componentName),
+ PackageManager.GET_META_DATA, userId);
+
+ if (services.isEmpty()) {
+ Slog.e(TAG, "No IME found");
+ return null;
+ }
+
+ if (services.size() > 1) {
+ Slog.e(TAG, "Only one IME service is supported.");
+ return null;
+ }
+
+ final ResolveInfo ri = services.get(0);
+ ServiceInfo si = ri.serviceInfo;
+ final String imeId = InputMethodInfo.computeId(ri);
+ if (!android.Manifest.permission.BIND_INPUT_METHOD.equals(si.permission)) {
+ Slog.e(TAG, imeId + " must have required"
+ + android.Manifest.permission.BIND_INPUT_METHOD);
+ return null;
+ }
+
+ if (!Build.IS_DEBUGGABLE && (si.applicationInfo.flags & ApplicationInfo.FLAG_SYSTEM) == 0) {
+ Slog.e(TAG, imeId + " must be pre-installed when Build.IS_DEBUGGABLE is false");
+ return null;
+ }
+
+ try {
+ return new InputMethodInfo(context, ri);
+ } catch (Exception e) {
+ Slog.wtf(TAG, "Unable to load input method " + imeId, e);
+ }
+ return null;
+ }
+
+ /**
+ * Manages the mapping rule from user ID to {@link InputMethodInfo}.
+ */
+ private static final class UserToInputMethodInfoMap {
+ @GuardedBy("mArray")
+ private final SparseArray<InputMethodInfo> mArray = new SparseArray<>();
+
+ @AnyThread
+ void put(@UserIdInt int userId, InputMethodInfo imi) {
+ synchronized (mArray) {
+ mArray.put(userId, imi);
+ }
+ }
+
+ @AnyThread
+ void remove(@UserIdInt int userId) {
+ synchronized (mArray) {
+ mArray.remove(userId);
+ }
+ }
+
+ @AnyThread
+ @Nullable
+ InputMethodInfo get(@UserIdInt int userId) {
+ synchronized (mArray) {
+ return mArray.get(userId);
+ }
+ }
+
+ @AnyThread
+ List<InputMethodInfo> getAsList(@UserIdInt int userId) {
+ final InputMethodInfo info = get(userId);
+ if (info == null) {
+ return Collections.emptyList();
+ }
+ return Collections.singletonList(info);
+ }
+ }
+
+ /**
+ * Takes care of IPCs exposed to the multi-client IME.
+ */
+ private static final class ImeCallbacks
+ extends IMultiClientInputMethodPrivilegedOperations.Stub {
+ private final PerUserData mPerUserData;
+ private final WindowManagerInternal mIWindowManagerInternal;
+
+ ImeCallbacks(PerUserData perUserData) {
+ mPerUserData = perUserData;
+ mIWindowManagerInternal = LocalServices.getService(WindowManagerInternal.class);
+ }
+
+ @BinderThread
+ @Override
+ public IBinder createInputMethodWindowToken(int displayId) {
+ synchronized (mPerUserData.mLock) {
+ // We assume the number of tokens would not be that large (up to 10 or so) hence
+ // linear search should be acceptable.
+ final int numTokens = mPerUserData.mDisplayIdToImeWindowTokenMap.size();
+ for (int i = 0; i < numTokens; ++i) {
+ final TokenInfo tokenInfo =
+ mPerUserData.mDisplayIdToImeWindowTokenMap.valueAt(i);
+ // Currently we issue up to one window token per display.
+ if (tokenInfo.mDisplayId == displayId) {
+ return tokenInfo.mToken;
+ }
+ }
+
+ final Binder token = new Binder();
+ Binder.withCleanCallingIdentity(
+ PooledLambda.obtainRunnable(WindowManagerInternal::addWindowToken,
+ mIWindowManagerInternal, token, TYPE_INPUT_METHOD, displayId));
+ mPerUserData.mDisplayIdToImeWindowTokenMap.add(new TokenInfo(token, displayId));
+ return token;
+ }
+ }
+
+ @BinderThread
+ @Override
+ public void deleteInputMethodWindowToken(IBinder token) {
+ synchronized (mPerUserData.mLock) {
+ // We assume the number of tokens would not be that large (up to 10 or so) hence
+ // linear search should be acceptable.
+ final int numTokens = mPerUserData.mDisplayIdToImeWindowTokenMap.size();
+ for (int i = 0; i < numTokens; ++i) {
+ final TokenInfo tokenInfo =
+ mPerUserData.mDisplayIdToImeWindowTokenMap.valueAt(i);
+ if (tokenInfo.mToken == token) {
+ mPerUserData.mDisplayIdToImeWindowTokenMap.remove(tokenInfo);
+ break;
+ }
+ }
+ }
+ }
+
+ @BinderThread
+ @Override
+ public void acceptClient(int clientId, IInputMethodSession inputMethodSession,
+ IMultiClientInputMethodSession multiSessionInputMethodSession,
+ InputChannel writeChannel) {
+ synchronized (mPerUserData.mLock) {
+ final InputMethodClientInfo clientInfo =
+ mPerUserData.getClientFromIdLocked(clientId);
+ if (clientInfo == null) {
+ Slog.e(TAG, "Unknown clientId=" + clientId);
+ return;
+ }
+ switch (clientInfo.mState) {
+ case InputMethodClientState.WAITING_FOR_IME_SESSION:
+ try {
+ clientInfo.mClient.setActive(true, false);
+ } catch (RemoteException e) {
+ // TODO(yukawa): Remove this client.
+ return;
+ }
+ clientInfo.mState = InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT;
+ clientInfo.mWriteChannel = writeChannel;
+ clientInfo.mInputMethodSession = inputMethodSession;
+ clientInfo.mMSInputMethodSession = multiSessionInputMethodSession;
+ break;
+ default:
+ Slog.e(TAG, "Unexpected state=" + clientInfo.mState);
+ break;
+ }
+ }
+ }
+
+ @BinderThread
+ @Override
+ public void reportImeWindowTarget(int clientId, int targetWindowHandle,
+ IBinder imeWindowToken) {
+ synchronized (mPerUserData.mLock) {
+ final InputMethodClientInfo clientInfo =
+ mPerUserData.getClientFromIdLocked(clientId);
+ if (clientInfo == null) {
+ Slog.e(TAG, "Unknown clientId=" + clientId);
+ return;
+ }
+ for (WindowInfo windowInfo : clientInfo.mWindowMap.values()) {
+ if (windowInfo.mWindowHandle == targetWindowHandle) {
+ final IBinder targetWindowToken = windowInfo.mWindowToken;
+ // TODO(yukawa): Report targetWindowToken and targetWindowToken to WMS.
+ if (DEBUG) {
+ Slog.v(TAG, "reportImeWindowTarget"
+ + " clientId=" + clientId
+ + " imeWindowToken=" + imeWindowToken
+ + " targetWindowToken=" + targetWindowToken);
+ }
+ }
+ }
+ // not found.
+ }
+ }
+
+ @BinderThread
+ @Override
+ public boolean isUidAllowedOnDisplay(int displayId, int uid) {
+ return mIWindowManagerInternal.isUidAllowedOnDisplay(displayId, uid);
+ }
+ }
+
+ /**
+ * Takes care of IPCs exposed to the IME client.
+ */
+ private static final class ApiCallbacks extends IInputMethodManager.Stub {
+ private final UserDataMap mUserDataMap;
+ private final UserToInputMethodInfoMap mInputMethodInfoMap;
+ private final AppOpsManager mAppOpsManager;
+ private final WindowManagerInternal mWindowManagerInternal;
+
+ ApiCallbacks(Context context, UserDataMap userDataMap,
+ UserToInputMethodInfoMap inputMethodInfoMap) {
+ mUserDataMap = userDataMap;
+ mInputMethodInfoMap = inputMethodInfoMap;
+ mAppOpsManager = context.getSystemService(AppOpsManager.class);
+ mWindowManagerInternal = LocalServices.getService(WindowManagerInternal.class);
+ }
+
+ @AnyThread
+ private boolean checkFocus(int uid, int pid, int displayId) {
+ return mWindowManagerInternal.isInputMethodClientFocus(uid, pid, displayId);
+ }
+
+ @BinderThread
+ @Override
+ public void addClient(IInputMethodClient client, IInputContext inputContext,
+ int selfReportedDisplayId) {
+ final int callingUid = Binder.getCallingUid();
+ final int callingPid = Binder.getCallingPid();
+ final int userId = UserHandle.getUserId(callingUid);
+ final PerUserData data = mUserDataMap.get(userId);
+ if (data == null) {
+ Slog.e(TAG, "addClient() from unknown userId=" + userId
+ + " uid=" + callingUid + " pid=" + callingPid);
+ return;
+ }
+ synchronized (data.mLock) {
+ data.addClientLocked(callingUid, callingPid, client, selfReportedDisplayId);
+ }
+ }
+
+ @BinderThread
+ @Override
+ public List<InputMethodInfo> getInputMethodList() {
+ return mInputMethodInfoMap.getAsList(UserHandle.getUserId(Binder.getCallingUid()));
+ }
+
+ @BinderThread
+ @Override
+ public List<InputMethodInfo> getVrInputMethodList() {
+ reportNotSupported();
+ return Collections.emptyList();
+ }
+
+ @BinderThread
+ @Override
+ public List<InputMethodInfo> getEnabledInputMethodList() {
+ return mInputMethodInfoMap.getAsList(UserHandle.getUserId(Binder.getCallingUid()));
+ }
+
+ @BinderThread
+ @Override
+ public List<InputMethodSubtype> getEnabledInputMethodSubtypeList(String imiId,
+ boolean allowsImplicitlySelectedSubtypes) {
+ reportNotSupported();
+ return Collections.emptyList();
+ }
+
+ @BinderThread
+ @Override
+ public InputMethodSubtype getLastInputMethodSubtype() {
+ reportNotSupported();
+ return null;
+ }
+
+ @BinderThread
+ @Override
+ public List getShortcutInputMethodsAndSubtypes() {
+ reportNotSupported();
+ return null;
+ }
+
+ @BinderThread
+ @Override
+ public boolean showSoftInput(
+ IInputMethodClient client, int flags, ResultReceiver resultReceiver) {
+ final int callingUid = Binder.getCallingUid();
+ final int callingPid = Binder.getCallingPid();
+ final int userId = UserHandle.getUserId(callingUid);
+ final PerUserData data = mUserDataMap.get(userId);
+ if (data == null) {
+ Slog.e(TAG, "showSoftInput() from unknown userId=" + userId
+ + " uid=" + callingUid + " pid=" + callingPid);
+ return false;
+ }
+ synchronized (data.mLock) {
+ final InputMethodClientInfo clientInfo = data.getClientLocked(client);
+ if (clientInfo == null) {
+ Slog.e(TAG, "showSoftInput. client not found. ignoring.");
+ return false;
+ }
+ if (clientInfo.mUid != callingUid) {
+ Slog.e(TAG, "Expected calling UID=" + clientInfo.mUid
+ + " actual=" + callingUid);
+ return false;
+ }
+ switch (clientInfo.mState) {
+ case InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT:
+ case InputMethodClientState.ALREADY_SENT_BIND_RESULT:
+ try {
+ clientInfo.mMSInputMethodSession.showSoftInput(flags, resultReceiver);
+ } catch (RemoteException e) {
+ }
+ break;
+ default:
+ if (DEBUG) {
+ Slog.e(TAG, "Ignoring showSoftInput(). clientState="
+ + clientInfo.mState);
+ }
+ break;
+ }
+ return true;
+ }
+ }
+
+ @BinderThread
+ @Override
+ public boolean hideSoftInput(
+ IInputMethodClient client, int flags, ResultReceiver resultReceiver) {
+ final int callingUid = Binder.getCallingUid();
+ final int callingPid = Binder.getCallingPid();
+ final int userId = UserHandle.getUserId(callingUid);
+ final PerUserData data = mUserDataMap.get(userId);
+ if (data == null) {
+ Slog.e(TAG, "hideSoftInput() from unknown userId=" + userId
+ + " uid=" + callingUid + " pid=" + callingPid);
+ return false;
+ }
+ synchronized (data.mLock) {
+ final InputMethodClientInfo clientInfo = data.getClientLocked(client);
+ if (clientInfo == null) {
+ return false;
+ }
+ if (clientInfo.mUid != callingUid) {
+ Slog.e(TAG, "Expected calling UID=" + clientInfo.mUid
+ + " actual=" + callingUid);
+ return false;
+ }
+ switch (clientInfo.mState) {
+ case InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT:
+ case InputMethodClientState.ALREADY_SENT_BIND_RESULT:
+ try {
+ clientInfo.mMSInputMethodSession.hideSoftInput(flags, resultReceiver);
+ } catch (RemoteException e) {
+ }
+ break;
+ default:
+ if (DEBUG) {
+ Slog.e(TAG, "Ignoring hideSoftInput(). clientState="
+ + clientInfo.mState);
+ }
+ break;
+ }
+ return true;
+ }
+ }
+
+ @BinderThread
+ @Override
+ public InputBindResult startInputOrWindowGainedFocus(
+ @StartInputReason int startInputReason,
+ @Nullable IInputMethodClient client,
+ @Nullable IBinder windowToken,
+ @StartInputFlags int startInputFlags,
+ @SoftInputModeFlags int softInputMode,
+ int windowFlags,
+ @Nullable EditorInfo editorInfo,
+ @Nullable IInputContext inputContext,
+ @MissingMethodFlags int missingMethods,
+ int unverifiedTargetSdkVersion) {
+ final int callingUid = Binder.getCallingUid();
+ final int callingPid = Binder.getCallingPid();
+ final int userId = UserHandle.getUserId(callingUid);
+
+ if (client == null) {
+ return InputBindResult.INVALID_CLIENT;
+ }
+
+ final boolean packageNameVerified =
+ editorInfo != null && InputMethodUtils.checkIfPackageBelongsToUid(
+ mAppOpsManager, callingUid, editorInfo.packageName);
+ if (editorInfo != null && !packageNameVerified) {
+ Slog.e(TAG, "Rejecting this client as it reported an invalid package name."
+ + " uid=" + callingUid + " package=" + editorInfo.packageName);
+ return InputBindResult.INVALID_PACKAGE_NAME;
+ }
+
+ final PerUserData data = mUserDataMap.get(userId);
+ if (data == null) {
+ Slog.e(TAG, "startInputOrWindowGainedFocus() from unknown userId=" + userId
+ + " uid=" + callingUid + " pid=" + callingPid);
+ return InputBindResult.INVALID_USER;
+ }
+
+ synchronized (data.mLock) {
+ final InputMethodClientInfo clientInfo = data.getClientLocked(client);
+ if (clientInfo == null) {
+ return InputBindResult.INVALID_CLIENT;
+ }
+ if (clientInfo.mUid != callingUid) {
+ Slog.e(TAG, "Expected calling UID=" + clientInfo.mUid
+ + " actual=" + callingUid);
+ return InputBindResult.INVALID_CLIENT;
+ }
+
+ switch (data.mState) {
+ case PerUserState.USER_LOCKED:
+ case PerUserState.SERVICE_NOT_QUERIED:
+ case PerUserState.SERVICE_RECOGNIZED:
+ case PerUserState.WAITING_SERVICE_CONNECTED:
+ case PerUserState.UNBIND_CALLED:
+ return InputBindResult.IME_NOT_CONNECTED;
+ case PerUserState.SERVICE_CONNECTED:
+ // OK
+ break;
+ default:
+ Slog.wtf(TAG, "Unexpected state=" + data.mState);
+ return InputBindResult.IME_NOT_CONNECTED;
+ }
+
+ WindowInfo windowInfo = null;
+ if (windowToken != null) {
+ windowInfo = clientInfo.mWindowMap.get(windowToken);
+ if (windowInfo == null) {
+ windowInfo = new WindowInfo(windowToken, WindowHandleSource.getNext());
+ clientInfo.mWindowMap.put(windowToken, windowInfo);
+ }
+ }
+
+ if (!checkFocus(clientInfo.mUid, clientInfo.mPid,
+ clientInfo.mSelfReportedDisplayId)) {
+ return InputBindResult.NOT_IME_TARGET_WINDOW;
+ }
+
+ if (editorInfo == null) {
+ // So-called dummy InputConnection scenario. For app compatibility, we still
+ // notify this to the IME.
+ switch (clientInfo.mState) {
+ case InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT:
+ case InputMethodClientState.ALREADY_SENT_BIND_RESULT:
+ final int windowHandle = windowInfo != null
+ ? windowInfo.mWindowHandle
+ : MultiClientInputMethodServiceDelegate.INVALID_WINDOW_HANDLE;
+ try {
+ clientInfo.mMSInputMethodSession.startInputOrWindowGainedFocus(
+ inputContext, missingMethods, editorInfo, startInputFlags,
+ softInputMode, windowHandle);
+ } catch (RemoteException e) {
+ }
+ break;
+ }
+ return InputBindResult.NULL_EDITOR_INFO;
+ }
+
+ switch (clientInfo.mState) {
+ case InputMethodClientState.REGISTERED:
+ case InputMethodClientState.WAITING_FOR_IME_SESSION:
+ clientInfo.mBindingSequence++;
+ if (clientInfo.mBindingSequence < 0) {
+ clientInfo.mBindingSequence = 0;
+ }
+ return new InputBindResult(
+ InputBindResult.ResultCode.SUCCESS_WAITING_IME_SESSION,
+ null, null, data.mCurrentInputMethodInfo.getId(),
+ clientInfo.mBindingSequence);
+ case InputMethodClientState.READY_TO_SEND_FIRST_BIND_RESULT:
+ case InputMethodClientState.ALREADY_SENT_BIND_RESULT:
+ clientInfo.mBindingSequence++;
+ if (clientInfo.mBindingSequence < 0) {
+ clientInfo.mBindingSequence = 0;
+ }
+ // Successful start input.
+ final int windowHandle = windowInfo != null
+ ? windowInfo.mWindowHandle
+ : MultiClientInputMethodServiceDelegate.INVALID_WINDOW_HANDLE;
+ try {
+ clientInfo.mMSInputMethodSession.startInputOrWindowGainedFocus(
+ inputContext, missingMethods, editorInfo, startInputFlags,
+ softInputMode, windowHandle);
+ } catch (RemoteException e) {
+ }
+ clientInfo.mState = InputMethodClientState.ALREADY_SENT_BIND_RESULT;
+ return new InputBindResult(
+ InputBindResult.ResultCode.SUCCESS_WITH_IME_SESSION,
+ clientInfo.mInputMethodSession,
+ clientInfo.mWriteChannel.dup(),
+ data.mCurrentInputMethodInfo.getId(),
+ clientInfo.mBindingSequence);
+ case InputMethodClientState.UNREGISTERED:
+ Slog.e(TAG, "The client is already unregistered.");
+ return InputBindResult.INVALID_CLIENT;
+ }
+ }
+ return null;
+ }
+
+ @BinderThread
+ @Override
+ public void showInputMethodPickerFromClient(
+ IInputMethodClient client, int auxiliarySubtypeMode) {
+ reportNotSupported();
+ }
+
+ @BinderThread
+ @Override
+ public void showInputMethodAndSubtypeEnablerFromClient(
+ IInputMethodClient client, String inputMethodId) {
+ reportNotSupported();
+ }
+
+ @BinderThread
+ @Override
+ public boolean isInputMethodPickerShownForTest() {
+ reportNotSupported();
+ return false;
+ }
+
+ @BinderThread
+ @Override
+ public void setInputMethod(IBinder token, String id) {
+ reportNotSupported();
+ }
+
+ @BinderThread
+ @Override
+ public void setInputMethodAndSubtype(IBinder token, String id, InputMethodSubtype subtype) {
+ reportNotSupported();
+ }
+
+ @BinderThread
+ @Override
+ public void registerSuggestionSpansForNotification(SuggestionSpan[] suggestionSpans) {
+ reportNotSupported();
+ }
+
+ @BinderThread
+ @Override
+ public boolean notifySuggestionPicked(
+ SuggestionSpan span, String originalString, int index) {
+ reportNotSupported();
+ return false;
+ }
+
+ @BinderThread
+ @Override
+ public InputMethodSubtype getCurrentInputMethodSubtype() {
+ reportNotSupported();
+ return null;
+ }
+
+ @BinderThread
+ @Override
+ public boolean setCurrentInputMethodSubtype(InputMethodSubtype subtype) {
+ reportNotSupported();
+ return false;
+ }
+
+ @BinderThread
+ @Override
+ public boolean switchToPreviousInputMethod(IBinder token) {
+ reportNotSupported();
+ return false;
+ }
+
+ @BinderThread
+ @Override
+ public boolean switchToNextInputMethod(IBinder token, boolean onlyCurrentIme) {
+ reportNotSupported();
+ return false;
+ }
+
+ @BinderThread
+ @Override
+ public void setAdditionalInputMethodSubtypes(String imiId, InputMethodSubtype[] subtypes) {
+ reportNotSupported();
+ }
+
+ @BinderThread
+ @Override
+ public int getInputMethodWindowVisibleHeight() {
+ reportNotSupported();
+ return 0;
+ }
+
+ @BinderThread
+ @Override
+ public void onShellCommand(@Nullable FileDescriptor in, @Nullable FileDescriptor out,
+ @Nullable FileDescriptor err, String[] args, @Nullable ShellCallback callback,
+ ResultReceiver resultReceiver) {
+ }
+ }
+}
diff --git a/services/core/java/com/android/server/inputmethod/multi-client-ime.md b/services/core/java/com/android/server/inputmethod/multi-client-ime.md
new file mode 100644
index 0000000..3021d2f
--- /dev/null
+++ b/services/core/java/com/android/server/inputmethod/multi-client-ime.md
@@ -0,0 +1,192 @@
+<!-- Copyright (C) 2018 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Multi Client Input Method Editors
+
+## History of Multi Client Input Method Editors (Multi Client IMEs)
+
+An advanced multi-display support is requested for certain Android form-factors so that user(s) can type text on each display at the same time without losing software keyboard focus in other displays (hereafter called "multi-client scenario"). This is not possible in Android IMEs built on top of `InputMethodService` class. The assumption that a single IME client can be focused at the same time was made before Android IME APIs were introduced in Android 1.5 and many public APIs in `InputMethodService` have already relied heavily on that assumption (hereafter called "single-client scenario"). Updating `InputMethodService` class to support multi-client scenario is, however, quite challenging because:
+
+ 1. doing so would introduce an unacceptable amount of complexity into `InputMethodService`, which is already hard to maintain,
+ 2. IME developers still need to update their implementation to be able to support parallel requests from multiple focused IME client, which may require non-trivial redesign in their side (e.g. input decoder, typing history database, ...), and
+ 3. actual use cases for multi IME clients are expected to be evolved rapidly hence the new protocol is not yet stable and not yet ready to be exposed as public APIs.
+
+Thus the first decision we made was that to support such special multi-display environments a new type of IME (hereafter called "multi-client IME") needs to be designed and implemented rather than reusing `InputMethodService` public class. On top of this decision, following decisions were also made:
+
+ * Multi-client IME V1 will be built on top of private APIs. This means:
+ * Multi-client IME must be pre-installed into the system. They cannot be distributed via application store since protocol compatibility is not guaranteed across devices and releases.
+ * The system should trust multi-client IME to some extent. System integrators are responsible for making sure that the pre-installed multi-client IME works as expected.
+ * Unlike `InputMethodService`, multiple multi-client IMEs cannot be enabled. The system pre-installs only one multi-client IME.
+ * Punt some special features of Android IMEs (e.g. fullscreen mode, InputMethodSubtype, ...) from V1 goal unless someone actually requests those features for multi-client IME scenario.
+ * Introduce `MultiClientInputMethodManagerService` (MCIMMS) for multi-client IME scenario and use it instead of `InputMethodManagerService` (IMMS) when a certain runtime flag is enabled at the device boot time. This means:
+ * basically no risk for single-client scenario,
+ * the feature can be easily deprecated, and
+ * it forces us to rewrite IME system server, which is expected to be a good chance to reconsider what Android IME protocol should look like.
+ * Most of form-factors such as Phones and TVs continue to use IMMS and support at most one focused IME client even under multi-display environment.
+
+
+## How to test
+
+On AOSP-based development devices (e.g. phones) where `android.os.Build.IS_DEBUGGABLE` returns `true` and you can have root access, you can enable multi-client IME feature by setting a valid component name that supports multi-client IME protocol to the system property `persist.debug.multi_client_ime`. Reboot is required for this to take effect.
+
+```shell
+# Build and install a sample multi-client IME
+make -j MultiClientInputMethod
+adb install -r $OUT/system/priv-app/MultiClientInputMethod/MultiClientInputMethod.apk
+
+# Enable multi-client IME for the side-loaded sample multi-client IME
+adb root
+adb shell setprop persist.debug.multi_client_ime com.example.android.multiclientinputmethod/.MultiClientInputMethod
+adb reboot
+```
+
+To disable multi-client IME on non-supported devices again, just clear `persist.debug.multi_client_ime` as follows. Reboot is still required for this to take effect.
+
+```shell
+# Disable multi-client IME again
+adb root
+adb shell "setprop persist.debug.multi_client_ime ''"
+adb reboot
+```
+
+## How to develop multi-client IMEs
+
+There is a sample multi-client IME in `development/samples/MultiClientInputMethod/`.
+
+## Versioning
+
+Neither forward nor backward compatibility is guaranteed in multi-client IME APIs. The system integrator is responsible for making sure that both the system and pre-installed multi-client IME are compatible with each other every time the system image is updated.
+
+## Implementation note
+
+### Unsupported features
+
+ * VR IME
+ * `VrManager#setVrInputMethod()` system API is not supported.
+ * InputMethodSubtype
+ * Following APIs are not supported
+ * `InputMethodManager#getEnabledInputMethodSubtypeList()`
+ * `InputMethodManager#getCurrentInputMethodSubtype()`
+ * `InputMethodManager#setCurrentInputMethodSubtype()`
+ * `InputMethodManager#getShortcutInputMethodsAndSubtypes()`
+ * `InputMethodManager#setAdditionalInputMethodSubtypes()`
+ * `InputMethodManager#getLastInputMethodSubtype()`
+ * `Settings.Secure#SELECTED_INPUT_METHOD_SUBTYPE`
+ * IME switching
+ * Following APIs are not supported
+ * `InputMethodManager#showInputMethodPicker()`
+ * `InputMethodManager#showInputMethodAndSubtypeEnabler()`
+ * `InputMethodManager#setInputMethod()`
+ * `InputMethodManager#setInputMethodAndSubtype()`
+ * `InputMethodManager#switchToLastInputMethod()`
+ * `InputMethodManager#switchToNextInputMethod()`
+ * `InputMethodManager#shouldOfferSwitchingToNextInputMethod()`
+ * `Settings.Secure#DEFAULT_INPUT_METHOD`
+ * `Settings.Secure#ENABLED_INPUT_METHODS`
+ * Direct-boot aware multi-client IME
+ * Device manufacturer can work around this by integrating in-app keyboard into the initial unlock screen.
+ * Full-screen mode
+ * Following API always returns `false`.
+ * `InputMethodManager#isFullscreenMode()`
+ * Custom inset
+ * For instance, floating IME cannot be implemented right now.
+ * Custom touchable region (`InputMethodService.Insets#touchableRegion`)
+ * Image Insertion API
+ * `InputConnection#commitContent()` API is silently ignored.
+ * `adb shell dumpsys` does not include any log from MCIMMS yet.
+
+### Security
+
+#### Root permission is required to enable MCIMMS on non-supported devices
+
+In order to override `persist.debug.multi_client_ime` device property, an explicit root permission is needed.
+
+#### Multi-client IME must be pre-installed
+
+Multi-client IME must be pre-installed since it is considered as part of the system component. This is verified by checking `ApplicationInfo.FLAG_SYSTEM` bit. This security check can be bypassed when `Build.IS_DEBUGGABLE` is `true` so that IME developers can easily side-load their APKs during development phase.
+
+```java
+public final class MultiClientInputMethodManagerService {
+ ...
+ @Nullable
+ private static InputMethodInfo queryInputMethod(Context context, @UserIdInt int userId,
+ @Nullable ComponentName componentName) {
+
+ ...
+
+ if (! && (si.applicationInfo.flags & ApplicationInfo.FLAG_SYSTEM) == 0) {
+ Slog.e(TAG, imeId + " must be pre-installed when Build.IS_DEBUGGABLE is false");
+ return null;
+ }
+```
+[services/core/java/com/android/server/inputmethod/MultiClientInputMethodManagerService.java](MultiClientInputMethodManagerService.java)
+
+
+#### Integer handle vs IBinder token
+
+Sometimes MCIMMS needs to issue certain types of identifiers to the multi-client IME so that the IME can later specify to which entity or resource it intends to access. A good example is the IME client identifier. Multi-client IME definitely need to be able to specify which IME client to be interacted with for certain operations. The problem is that MCIMMS cannot simply pass `IInputMethodClient` to the multi-client IME as an ID because it would allow the IME to make IPC calls to the IME client. For this kind of situations, we usually use `Binder` object just as a non-spoofable token. For instance, IMMS creates another 'Binder' token then pass it to the IME, instead of directly passing 'IWindow' Binder token.
+
+```java
+public class InputMethodManagerService extends IInputMethodManager.Stub
+ implements ServiceConnection, Handler.Callback {
+ ...
+ @GuardedBy("mMethodMap")
+ private final WeakHashMap<IBinder, IBinder> mImeTargetWindowMap = new WeakHashMap<>();
+
+ ...
+
+ @GuardedBy("mMethodMap")
+ @NonNull
+ InputBindResult attachNewInputLocked(@StartInputReason int startInputReason, boolean initial) {
+ ...
+ final Binder startInputToken = new Binder();
+ final StartInputInfo info = new StartInputInfo(mCurToken, mCurId, startInputReason,
+ !initial, mCurFocusedWindow, mCurAttribute, mCurFocusedWindowSoftInputMode,
+ mCurSeq);
+ mImeTargetWindowMap.put(startInputToken, mCurFocusedWindow);
+ ...
+ }
+
+ ...
+
+ @BinderThread
+ private void reportStartInput(IBinder token, IBinder startInputToken) {
+ if (!calledWithValidToken(token)) {
+ return;
+ }
+
+ synchronized (mMethodMap) {
+ final IBinder targetWindow = mImeTargetWindowMap.get(startInputToken);
+ if (targetWindow != null && mLastImeTargetWindow != targetWindow) {
+ mWindowManagerInternal.updateInputMethodTargetWindow(token, targetWindow);
+ }
+ mLastImeTargetWindow = targetWindow;
+ }
+ }
+```
+[services/core/java/com/android/server/inputmethod/InputMethodManagerService.java](InputMethodManagerService.java)
+
+However, in MCIMMS, for certain cases we decided to use a simple integer token, which can be spoofable and can be messed up if integer overflow happens. This is because:
+
+ * It does not make much sense to worry about malicious multi-client IMEs, because it is guaranteed to be a pre-installed system component.
+ * Integer token is expected to be a more lightweight that `Binder` token.
+ * For that use case, integer overflow is unrealistic.
+ * Strict user separation is still enforced. Multi-client IMEs are still not allowed to interact with other users' resources by any means.
+
+Currently the following IDs are implemented as integer tokens:
+
+ * Client ID
+ * Window Handle
+ * Note that each IME client has its own Window Handle mapping table. Window Handle is valid only within the associated IME client.
diff --git a/services/java/com/android/server/SystemServer.java b/services/java/com/android/server/SystemServer.java
index bbd1e97..86eb6f3 100644
--- a/services/java/com/android/server/SystemServer.java
+++ b/services/java/com/android/server/SystemServer.java
@@ -89,6 +89,7 @@
import com.android.server.hdmi.HdmiControlService;
import com.android.server.input.InputManagerService;
import com.android.server.inputmethod.InputMethodManagerService;
+import com.android.server.inputmethod.MultiClientInputMethodManagerService;
import com.android.server.job.JobSchedulerService;
import com.android.server.lights.LightsService;
import com.android.server.media.MediaResourceMonitorService;
@@ -1016,7 +1017,12 @@
// Bring up services needed for UI.
if (mFactoryTestMode != FactoryTest.FACTORY_TEST_LOW_LEVEL) {
traceBeginAndSlog("StartInputMethodManagerLifecycle");
- mSystemServiceManager.startService(InputMethodManagerService.Lifecycle.class);
+ if (MultiClientInputMethodManagerService.isConfiguredToUse()) {
+ mSystemServiceManager.startService(
+ MultiClientInputMethodManagerService.Lifecycle.class);
+ } else {
+ mSystemServiceManager.startService(InputMethodManagerService.Lifecycle.class);
+ }
traceEnd();
traceBeginAndSlog("StartAccessibilityManagerService");