Nylas API architecture

The diagram below represents how Beamery (your application in diagram) uses the Nylas API to support synchronization with various email providers including Gmail, Outlook/Microsoft Exchange, and Exchange ActiveSync.

Your application uses Nylas to send requests to create, read, update, and delete objects from the provider. Nylas also monitors the provider for changes on each authenticated account, and sends those changes to your application so it can send notifications and react appropriately.

Authentication Steps - summary overview:

1. Redirect the user to Nylas Hosted Auth service

2. User consent auth: The user will log in using their email provider and consent to the permissions requested. For Exchange ActiveSync, the process is similar, with users entering their credentials and granting necessary permissions.

3. Handle the User Redirect and Auth Credentials

Auth process diagram:

Step 1: Redirect the User - Beamery application, redirect the user to the Nylas Hosted authentication service, and include query parameters that specify scopes, a redirect URL, and the authorization flow.

Step 2: User Consent - The user will log in using the email provider and consent to the permissions requested. Nylas will automatically detect the provider and direct the user to the correct screen.

Step 3: Get the Access Token - Once the user has signed in, Nylas stores the credentials securely and then the Beamery service will retrieve the mailbox token which will be Encrypted via Google KMS service and stored in the Beamery database.

Microsoft O365 consideration for authentication

When attempting to authenticate with their mailbox for connected email/calendar, users may be prompted with a screen that says they need admin approval. If this happens, please refer to this guide to see the options available by your IT admin team in order to grant access in Azure.

Exchange ActiveSync

Beamery supports synchronization with Exchange ActiveSync. When setting up Exchange ActiveSync, follow the standard authentication steps. The user will enter their Exchange credentials and grant the necessary permissions. Once authenticated, Nylas handles the synchronization of email and calendar data between the Exchange ActiveSync account and Beamery.

Scopes

Beamery UI example - Settings → Email Permission page

The list of allowed scopes is present in the following table:

email.send

email.read_only

calendar

Microsoft Scopes

Scopes used in Beamery Azura Application:

Scopes used in Beamery Google Application:

The token security

The token is Google KMS service encoded and stored in Beamery Database. The token is used for synchronization processes and has access to assets which have been granted by the user ducting connection phase.

The required Microsoft Scopes

In the case of connected mailbox, Microsoft service based the next Microsoft Scopes used during authorization:

During mailbox connection, the customer could select the feature to grant access. At the moment this could be

• Emails

  • "email.send"

  • "email.read_only”

• Calendars (schedule pages)

  • "calendar"

The combination of required Microsoft Scopes depending on selected features are present in the following table:

Data Flow

How data is requested:

1. We use cron worker only for streaming mailboxes which have to be passed to “sync process”

2. Sync engine and all manipulations with DB and communication with Nylas api happen only in queue worker

3. The queue consumers can be easily increased to make it possible to sync mailboxes parallel

4. Nylas webhooks are be used to mark mailboxes which have new updates and could be triggered for “syncing”

5. Nylas webhook update property updatedAt for related mailbox. This property informs us that mailbox had new changes like message or thread - created/updated

6. The Nylas cron worker runs each 10min and streaming mailboxes which match next criteria:

   1. The last sync was done longer then 12h ago for mailbox

   2. updatedAt timestamp exists, which show us we have new updates for mailbox

   3. lockedAt - property not exists. this is flag show that mailbox already passed in queue to process

7. Each streamed mailbox is passed to queue for process and lockedAt is set

8. When sync done - mailbox properties: lockedAt and updatedAt unset and syncedAt updated with new timestamp

Nylas v (Upon mailbox connection) Diagram

Nylas Delta Sync (Continuous) Diagram

The Nylas Sync Engine builds a transaction log that records every change as it synchronizes users' mailboxes. Delta sync uses these changes, exposed through the Delta endpoint, to process new data quickly without fetching an index of the user's mailbox or performing a large number of API calls.

The initial delta cursor obtained during the initial Historical Sync process

For each asset (“thread“, “message“, “event“) we have our own cron worker:

Data Structure and Storage

Data that is not stored - but is synced on request:

Credentials

We do not store any credentials used to access and synchronise email, other than the name of the email address used for a mailbox.

Messages

The message data is synced but not stored in a DB.

1. We are storing *threads* ( IDs to allow us to retrieve messages on individual requests) in our DB.

the data we have access to is outlined here:

https://developer.nylas.com/docs/api/v2/#tag--Threads

2. When a user clicks on a message, thus “expanding” a thread, we are making API call to get email data to get messages related to the thread we are adding `thread_id` query parameter to GET request

`https://api.nylas.com/messages?thread_id=<THREAD_ID>’

The response is an Array of message documents:

https://developer.nylas.com/docs/api/v2/#tag--Messages

Reply event recognition

We are consistently syncing the messages without storing any data. The reason of syncing is looking to get exposes several additional RFC2822 headers.

We are using “In-Reply-To” id to recognise the “reply” event for an email send from a Beamery Campaign engine and replying directly to a connected mailbox.

Data we store (when retrieved for contacts in Beamery system)

The data below we have access via a connected mailbox Token. The structure of synced objects is converted based on Beamery requirements. The table below is a guideline based on a typical data structure.

Thread

The purpose for synchronisation: Inside contact timeline to show synchronised thread activities and retrieve messages of the thread from Nylas service:

Calendar Event

The purpose for synchronisation: Inside contact timeline to show calendar events activities (future / past) interaction:

What happens when a user leaves or disconnects their mailbox?

When a user is removed, or when they “disconnect” their mailbox, we :

• Remove all synced threads and calendar events

• revoke the mailbox token that needed to be able to establish a connection with the provider

Do I need to connect emails to send emails?

No. You can send and receive emails directly via Beamery (routed through our partner, Sendgrid), with replies tracked through beamery-generated reply addresses. To improve deliverability, this approach will typically involve setting up a ‘whitelabel’ for the domain, such that emails are sent via your domain. The process for this involves Beamery sending your IT team our email Cname. Your team then installs the cname, which will point to existing MX, SPF and DKIM records.

Is there a document which explains security on the Nylas side?

Yes - here is a link to the document

Please also review frequently asked questions in the page Beamery + Nylas - Connected Email FAQs