How to add support for rich content in the webview

Victor Omondi -

The TalkJS chat API has support for file attachments with various file formats allowed. We have shown in a previous tutorial how to enable file upload in the Android WebView. Another important feature would be enabling your users to upload images, videos, stickers and gifs from their keyboards, clipboard or through drag and drop. Android 12 (API Level 31) introduced a new unified API that simplifies how to receive rich content from the above sources. This tutorial will show you how to use the unified API in your app’s WebView implementation.

Prerequisites

For this tutorial we shall be using Kotlin. The concepts, types and method names are exactly the same as in Java with the differences only being in the syntax. Similarly, Android Studio and IntelliJ can convert Kotlin code to Java.

We will assume you already have an Android project setup in Android Studio or your favourite editor/IDE. Also make sure that you have the appropriate permission in your AndroidManifest.xml file.

<uses-permission android:name="android.permission.INTERNET" />

In the file containing your Activity class, we are going to define a constant:

private val MIME_TYPES = arrayOf("image/*", "video/*")

This array contains the mime types that our application is gonna support in reference to receiving rich content. Modify this depending on your application’s use case.

In your Activity class, you’ll need one field called html that contains the html document that will be loaded in the WebView. We are hardcoding it as a string just to simplify the code we’ll need to implement. Don’t forget to replace the string, YOUR_APP_ID, with your TalkJS app ID from the dashboard.

private val html = """
  <!DOCTYPE html>
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <script>
      (function(t,a,l,k,j,s){
      s=a.createElement('script');s.async=1;s.src="https://cdn.talkjs.com/talk.js";a.head.appendChild(s)
      ;k=t.Promise;t.Talk={v:3,ready:{then:function(f){if(k)return new k(function(r,e){l.push([f,r,e])});l
      .push([f])},catch:function(){return k&&new k()},c:l}};})(window,document,[]);
     </script>
  </head>
  <body style="margin: 0px;">
    <div id="talkjs-container" style="width: 100%; height: 750px"></div>
  </body>
  <script type="module">
    await Talk.ready;
 
    const me = new Talk.User({
        id: '432156789',
        name: 'Sebastian',
        email: 'Sebastian@example.com',
        photoUrl: 'https://demo.talkjs.com/marketplace_demo/img/sebastian.jpg',
        welcomeMessage: null,
        role: 'default',
    });
 
    const other = new Talk.User({
        id: '123456789',
        name: 'Alice',
        email: 'alice@example.com',
        photoUrl: 'https://demo.talkjs.com/marketplace_demo/img/alice.jpg',
        welcomeMessage: null,
        role: 'default',
    });
 
    const talkSession = new Talk.Session({
       appId: 'YOUR_APP_ID',
       me: me,
    });
 
    const conversation = talkSession.getOrCreateConversation(Talk.oneOnOneId(me, other));
 
    conversation.setParticipant(me);
    conversation.setParticipant(other);
 
    window.chatbox = talkSession.createChatbox(conversation);
    window.chatbox.mount(document.getElementById('talkjs-container'));
 
    window.chatbox.select(conversation);
  </script>
 
  </html>
""".trimIndent()

Extending the WebView class

The key to implementing the unified API is through implementing the onReceiveContentListener interface. However, the default WebView’s implementation of onCreateInputConnection bypasses onReceiveContentListener. Therefore, we will need to extend the WebView class and override the onCreateInputConnection method.

class MyWebView : WebView {
 
   constructor(context: Context) : super(context)
   constructor(context: Context, attributeSet: AttributeSet) : super(context, attributeSet)
 
   override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
       val inputConnection = super.onCreateInputConnection(outAttrs)
       if (inputConnection == null) {
           return inputConnection
       }
 
       EditorInfoCompat.setContentMimeTypes(outAttrs, MIME_TYPES)
       return InputConnectionCompat.createWrapper(this, inputConnection, outAttrs)
   }
}

Calling the superclass method preserves the built-in behaviour (sending and receiving text) and gives you a reference to the InputConnection. setContentMimeTypes tells the Input Method Editor (the keyboard in most cases) which MIME types we support. The call to InputConnectionCompat.createWrapper is what enables us to eventually use onReceiveContentListener.

Implementing onReceiveContentListener

This interface has only one method, onReceiveContent, that we will implement. Our implementation will only handle content that have a URI and return all the other content to be handled through the default platform behaviour.

class MyReceiver() : OnReceiveContentListener {
 
   private fun getFileName(contentResolver: ContentResolver, uri: Uri): String? {
       val cursor = contentResolver.query(uri, null, null, null, null, null)
       return cursor?.use {
           val nameIndex = it.getColumnIndex(OpenableColumns.DISPLAY_NAME)
           it.moveToFirst()
 
           cursor.getString(nameIndex)
       }
   }
 
   override fun onReceiveContent(view: View, payload: ContentInfoCompat): ContentInfoCompat? {
       val split = payload.partition { item -> item.uri != null }
       val uriContent = split.first
       val remaining = split.second
 
       val clip = uriContent.clip
       if (clip.itemCount > 0) {
           val contentResolver = view.context.contentResolver
 
           val uri = clip.getItemAt(0).uri
           val mimeType = contentResolver.getType(uri)
           val fileName = getFileName(contentResolver, uri)
 
           val bufferedInputStream = contentResolver.openInputStream(uri)?.buffered()
           val javaScript = bufferedInputStream.use {
               val jsonByteArray = Json.encodeToString(it?.readBytes())
               """
               var byteArray = new Int8Array($jsonByteArray);
               var mediaFile = new File([byteArray], "$fileName", { type: "$mimeType" });
               window.chatbox.sendFile(mediaFile);
               """.trimIndent()
           }
 
           val myWebView = view as MyWebView
           myWebView.post {
               myWebView.evaluateJavascript(javaScript, null)
           }
       }
       return remaining
   }
}

The getFileName method is just a helper method to allow us to retrieve the name of a file given its URI and a ContentResolver.

The first part of onReceiveContent has us split the content into two separate variables: uriContent and remaining. These two variables contain content that have URI and those that do not respectively. They are both of type: ContentInfoCompat which is just a backward-compatible wrapper to ContentInfo.

We then retrieve the URI of the first item. You can easily change this implementation to loop through the contents and execute the subsequent code on each item’s URI.

The URI together with the ContentResolver enable us to retrieve the item’s MIME type and its file name. They also enable us to open a buffered input stream in order to read the actual bytes of data for that particular file. The array of bytes is encoded into a JSON string before being concatenated into a string that will be injected into the webview.

The JavaScript code that will be injected first creates an ArrayBuffer of type Int8Array. This ArrayBuffer together with the file name and MIME type are used to create a File object that will be uploaded to TalkJS via the Chatbox.sendFile method. This string is then injected into the WebView using the evaluateJavaScript method. The WebView requires that this method be called in the UI thread, hence the need for us to enqueue it using post.

Wrapping up

With all that in place, we now need to use our custom WebView implementation and also set its onReceiveContentListener. We will start by specifying it in our activity’s layout file. Change the package name to match where you defined your WebView implementation.

<com.talkjs.sample.MyWebView
   android:id="@+id/webView"
   android:layout_width="match_parent"
   android:layout_height="match_parent" />

We then need to update our Activity’s onCreate method. The implementation below uses View Binding.

private lateinit var binding: ActivityMainBinding
 
override fun onCreate(savedInstanceState: Bundle?) {
   super.onCreate(savedInstanceState)
   binding = ActivityMainBinding.inflate(layoutInflater)
   setContentView(binding.root)
 
   val webView = binding.webView
   ViewCompat.setOnReceiveContentListener(webView, MIME_TYPES, MyReceiver())
 
   webView.settings.javaScriptEnabled = true
   webView.loadDataWithBaseURL(
       "https://app.talkjs.com",
       html,
       "text/html",
       null,
       "https://app.talkjs.com"
   )
}

The important part is the call to setOnReceiveContentListener. The parameters it takes are our webview implementation, the MIME types we support and an instance of the class that implemented the onReceiveContentListener interface. Also note that we use the html variable defined earlier when calling loadDataWithBaseURL.

Your TalkJS users should now be able to send rich media content from their keyboards.

The code shown in this tutorial is available as an Android project on Github. You can open the project on Android Studio and run on your device or emulator.