How to export chat messages from TalkJS

We’ll show you how to export chat messages from a TalkJS chat. This should be useful for various use cases, like providing a chat transcript for a particular chat or serving as an audit trail for freelancer platforms like Upwork. In this post, let's see how to use the REST API provided by TalkJS to export messages using Nodejs.

See our GitHub repo for the example code for this tutorial.

Configuring the Project

This guide assumes that you have already configured a chat application using TalkJS and initialized a chat. If not, please refer to the getting started guide, set up a chat application, and create some messages.

We will utilize the Conversations endpoint of the TalkJS REST API to export chat messages. The node-fetch module is used to facilitate the API requests and the json2csv module to extract data from exported messages.

Install the dependencies

npm install node-fetch json2csv

Using the REST API to Export All Chat Messages

Messages are considered a sub-resource of Conversations. Therefore, we should target the specific conversation when interacting with the API to export chat messages. The API call for this export will be as follows.

GET https://api.talkjs.com/v1/{appId}/conversations/{conversationId}

If we know the specific conversation id, simply running a GET request with the specified id will return all the messages in that conversation. However, the above request needs to be executed for each available conversation if we need to export all messages from all conversations.

The following code block demonstrates how to export all the messages from all conversations.

The logic of that script can be broken down into two parts as follows.

1. Obtain the list of Conversations
2. Obtain the messages for each conversation

The exportMessages function acts as the primary function, which takes the user credentials and calls the listMessages function. First, get the list of conversations and then loop through the conversation list to obtain all the messages. The listMessages function acts as the intermediary that creates the API request depending on the provided parameters. Then it calls upon the makeRequest function to carry out the actual API call using node-fetch and return the response.

export-messages.js

import fetch from "node-fetch";
const host = "https://api.talkjs.com";
 
// Credentials
const appId = "<APP ID>";
const secretKey = "<Secret Key>";
 
// Export Messages
async function exportMessages() {
 // Get a List of Conversations
 let conversations = await getConversations();
 
 let allMessages = [];
 // Get each Message from the Conversations
 for (let conversation of conversations) {
   const messages = await getMessages({
     conversationId: conversation.id
   });
   allMessages.push(messages);
 }
 
 console.log(allMessages);
}
 
// Retrieve the specified resource
async function getResource({ resource, lastId, limit }) {
 const paginateMaybe = lastId ? `&startingAfter=${lastId}` : "";
 const path =
   host + `/v1/${appId}/${resource}?limit=${limit}${paginateMaybe}`;
 // Call makeRequest function and get the resource data
 let resources = await makeRequest({ path, verb: "GET" });
 if (resources.length === limit) {
   const [last] = resources.slice(-1);
   const newOnes = await getResource({
      resource,
      limit,
      lastId: last.id,
    });
   resources = [...resources, ...newOnes];
 }
 return resources;
}
 
function getConversations() {
 return getResource({
   resource: "conversations",
   limit: 30,
 });
}
 
function getMessages({ conversationId }) {
 return getResource({
   resource: `conversations/${conversationId}/messages`,
   limit: 100,
 });
}
 
// Make the GET Request to Obtain the Data
async function makeRequest({ path, verb, body }) {
 console.log("[API Request]", `Exporting Data: ${verb} ${path}.`);
 const response = await fetch(path, {
   method: verb,
   body: JSON.stringify(body),
   headers: {
     "Content-Type": "application/json",
     Authorization: `Bearer ${secretKey}`,
   },
 });
 
 await new Promise((resolve) => setTimeout(resolve, 250));
 
 if (response.status === 200) {
   const result = await response.json();
   console.log("[API Request]", `Export Completed: ${verb} ${path}.`);
 
   return result.data;
 } else {
   const error = await response.text();
   console.warn("[API Request]", `Error: ${error}, Path: ${verb} ${path}`);
   return [];
 }
}
 
exportMessages();

If we execute the above code block, it will look at all available conversations, obtain the messages, and print them as console output.

Result

Saving Exported Data as CSV

We can easily convert this data into a JSON string using the JSON.stringify function. Then we can write the resulting string to a file by adding the following code to the export-messages file and modifying the exportMessages function as shown below.

// new import for fs
import * as fs from 'fs';
 
async function exportMessages(source) {
    ........
 
    // Write to File as JSON
    let allMessagesJSON = JSON.stringify(allMessages);
    fs.writeFileSync(`all-messages-application-${appIdsource.id}.json`, allMessagesJSON);
 
}

To create a better readable data export, we can utilize the json2csv package, extract the relevant information from the exported dataset, and save it as a CSV file. Import the json2csv package, parse the messages to extract the relevant fields and save it as a CSV file.

// new import for fs and json2csv
import * as fs from 'fs';
import { Parser, transforms } from "json2csv";
 
// Export Messages
async function exportMessages(source) {
 try {
   // Get a List of Conversations
   let conversations = await getConversations();
 
   let allMessages = [];
   // Get each Message from the Conversations
   for (let conversation of conversations) {
     const messages = await getMessages({
       conversationId: conversation.id
     });
 
     const fields = [
       'conversationId',
       'createdAt',
       { label: "messageId", value: "id" },
       "text",
       "attachment.url",
       "attachment.size",
       "location",
       "custom",w
       "origin",
       "readBy",
       "senderId",
       "type"];
     const opts = {
       fields,
       transforms: transforms.flatten({separator: '_'}),
       quote: ''
     };
 
     // Extract the Data
     const parser = new Parser(opts);
     const messages_csv = parser.parse(messages);
     console.log("\n==== CSV Extract ====");
     console.log(messages_csv);
     console.log("\n");
 
     allMessages.push(messages_csv);
   }
 
   // Write to CSV File
   fs.writeFileSync(`all-messages-application-${source.id}.csv`, allMessages.toString());
 } catch (err) {
   console.error(err);
 }
}

Executing the modified code will result in a similar output to the following and create a CSV file named all-messages-application-<application id> containing all the messages.

CSV File

Likewise, you can export the chat messages from your conversations.