search
Connected Email & Calendar Overview (Nylas)

Last Updated:

deb0edff-05d3-40dc-b944-201c0a74dda2.png

 

Nylas API architecture

The diagram below represents how Beamery (your application in diagram) uses the Nylas API to support synchronization with various email providers including Gmail, Outlook/Microsoft Exchange, and Exchange ActiveSync.

Your application uses Nylas to send requests to create, read, update, and delete objects from the provider. Nylas also monitors the provider for changes on each authenticated account, and sends those changes to your application so it can send notifications and react appropriately.

An architecture diagram showing how Nylas interacts with your application and your end users' service providers.

Authentication Steps - summary overview:

  1. Redirect the user to Nylas Hosted Auth service

  2. User consent auth: The user will log in using their email provider and consent to the permissions requested. For Exchange ActiveSync, the process is similar, with users entering their credentials and granting necessary permissions.

  3. Handle the User Redirect and Auth Credentials

00a08694-67f1-430f-87c0-ec1bc03771ba.png

Auth process diagram:

7cd5df6b-0765-4a80-bd1c-e8f21ee1d21d.png

Step 1: Redirect the User - Beamery application, redirect the user to the Nylas Hosted authentication service, and include query parameters that specify scopes, a redirect URL, and the authorization flow.

Step 2: User Consent - The user will log in using the email provider and consent to the permissions requested. Nylas will automatically detect the provider and direct the user to the correct screen.

 

a3fde1db-b8d0-45a1-a17c-30545e85e87c.png

Step 3: Get the Access Token - Once the user has signed in, Nylas stores the credentials securely and then the Beamery service will retrieve the mailbox token which will be Encrypted via Google KMS service and stored in the Beamery database.

Microsoft O365 consideration for authentication

When attempting to authenticate with their mailbox for connected email/calendar, users may be prompted with a screen that says they need admin approval. If this happens, please refer to this guide to see the options available by your IT admin team in order to grant access in Azure.

Exchange ActiveSync

Beamery supports synchronization with Exchange ActiveSync. When setting up Exchange ActiveSync, follow the standard authentication steps. The user will enter their Exchange credentials and grant the necessary permissions. Once authenticated, Nylas handles the synchronization of email and calendar data between the Exchange ActiveSync account and Beamery.

Scopes

Beamery UI example - Settings → Email Permission page

7c296502-86a8-48aa-a42e-f714a2937e5a.png

The list of allowed scopes is present in the following table:

Scope

Feature

Overview

email.send

Send direct email

Send messages only. No read or modify privileges on users' emails.

email.read_only

Sync emails/campaigns tracking

Read all messages, threads, file attachments, drafts, and email metadata like headers—no write operations.

calendar

Scheduling

Read and modify calendars and events.

 

Microsoft Scopes

Scopes used in Beamery Azura Application:

Microsoft Scope

App Manifest

Description

offline_access

Microsoft Graph

Maintain access to data you have given it access to

openid

Microsoft Graph

Sign users in

profile

Microsoft Graph

View users' basic profile

User.Read

Microsoft Graph

Sign in and read user profile

User.Read

Azure Active Directory Graph

Sign in and read user profile

Calendar.Read.Shared

Microsoft Graph

Nylas supports Microsoft Graph for events only.

Calendar.ReadWrite.Shared

Microsoft Graph

Nylas supports Microsoft Graph for events only

 

Scopes used in Beamery Google Application:

Google Scope

Nylas Scopes

gmail.send

email.send

gmail.readonly

email.read_only. Includes gmail.metadata.

calendar

calendar

calendar.readonly

calendar.read_only

 

The token security

The token is Google KMS service encoded and stored in Beamery Database. The token is used for synchronization processes and has access to assets which have been granted by the user ducting connection phase.

06b538dc-119e-4e25-93dc-60c6e312d1cb.png

The required Microsoft Scopes

In the case of connected mailbox, Microsoft service based the next Microsoft Scopes used during authorization:

aaed9e97-8d95-456f-b4f5-1a92e8b51611.png

During mailbox connection, the customer could select the feature to grant access. At the moment this could be

  • Emails

    • "email.send"

    • "email.read_only”

  • Calendars (schedule pages)

    • "calendar"

The combination of required Microsoft Scopes depending on selected features are present in the following table:

 

Emails

Calendars

Emails/Calendars

Office 365

  • offline_access

  • opened

  • profile

  • User.Read

  • offline_access

  • opened

  • profile

  • User.Read

  • offline_access

  • opened

  • profile

  • User.Read

It's also possible to use the following set of scopes if you want to connect such account using Microsoft's Graph AP

 

  • offline_access

  • openid

  • profile

  • User.Read

  • Calendar.ReadWrite.Shared

 

 

Data Flow

How data is requested:

  1. We use cron worker only for streaming mailboxes which have to be passed to “sync process”

  2. Sync engine and all manipulations with DB and communication with Nylas api happen only in queue worker

  3. The queue consumers can be easily increased to make it possible to sync mailboxes parallel

  4. Nylas webhooks are be used to mark mailboxes which have new updates and could be triggered for “syncing”

  5. Nylas webhook update property updatedAt for related mailbox. This property informs us that mailbox had new changes like message or thread - created/updated

  6. The Nylas cron worker runs each 10min and streaming mailboxes which match next criteria:

    1. The last sync was done longer then 12h ago for mailbox

    2. updatedAt timestamp exists, which show us we have new updates for mailbox

    3. lockedAt - property not exists. this is flag show that mailbox already passed in queue to process

  7. Each streamed mailbox is passed to queue for process and lockedAt is set

  8. When sync done - mailbox properties: lockedAt and updatedAt unset and syncedAt updated with new timestamp

Nylas v (Upon mailbox connection) Diagram

e0c8b98e-0bd8-41cc-a90b-9991a8ae2593.png

Nylas Delta Sync (Continuous) Diagram

The Nylas Sync Engine builds a transaction log that records every change as it synchronizes users' mailboxes. Delta sync uses these changes, exposed through the Delta endpoint, to process new data quickly without fetching an index of the user's mailbox or performing a large number of API calls.

The initial delta cursor obtained during the initial Historical Sync process

For each asset (“thread“, “message“, “event“) we have our own cron worker:

Assets

Worker

Thread

nylas-deltas-thread

Events

nylas-deltas-others

Messages

nylas-deltas-message

 

dc51edcc-b95e-4bb5-aefb-0655d1938b42.png

Data Structure and Storage

Data that is not stored - but is synced on request:

Credentials

We do not store any credentials used to access and synchronise email, other than the name of the email address used for a mailbox.

Messages

The message data is synced but not stored in a DB.

1. We are storing *threads* ( IDs to allow us to retrieve messages on individual requests) in our DB.

the data we have access to is outlined here:

https://developer.nylas.com/docs/api/v2/#tag--Threads

2. When a user clicks on a message, thus “expanding” a thread, we are making API call to get email data to get messages related to the thread we are adding `thread_id` query parameter to GET request

`https://api.nylas.com/messages?thread_id=<THREAD_ID>’

The response is an Array of message documents:

https://developer.nylas.com/docs/api/v2/#tag--Messages

Reply event recognition

We are consistently syncing the messages without storing any data. The reason of syncing is looking to get exposes several additional RFC2822 headers.

Header

Requirement

Description

In-Reply-To

optional

The parent Message-Id to which this message replied. Expected null for messages that are not replies.

We are using “In-Reply-To” id to recognise the “reply” event for an email send from a Beamery Campaign engine and replying directly to a connected mailbox.

Data we store (when retrieved for contacts in Beamery system)

The data below we have access via a connected mailbox Token. The structure of synced objects is converted based on Beamery requirements. The table below is a guideline based on a typical data structure.

Thread

The purpose for synchronisation: Inside contact timeline to show synchronised thread activities and retrieve messages of the thread from Nylas service:

Attribute

Type

Description

id

string

Globally unique object identifier

account_id

string

Reference to parent account object

participants

array

List of participant objects computed from all messages in the thread. Keys are emails.

message_ids

array

Array of IDs for messages within the thread, sorted by their timestamps

has_attachments

boolean

Indicates if the thread has files attached to it.

 

Calendar Event

The purpose for synchronisation: Inside contact timeline to show calendar events activities (future / past) interaction:

Attribute

Type

Description

id

string

Globally unique object identifier

object

string

A string describing the type of object (value is "event")

account_id

string

Reference to parent account object

calendar_id

string

Reference to the parent calendar object

title

string

Title of the event, usually kept short.

description

string

Description of the event, which may contain more details or an agenda.

when

subobject

One of four subobjects corresponding to the time and duration of an event: time, timespan, date, or datespan. See below for more details.

location

string

Location, such as a physical address or meeting room name

owner

string

The owner of the event, usually their email or name and email.

participants

array

Array of others invited to the event. Keys are email, status.

status

string

Either confirmed, tentative, or cancelled.

read_only

boolean

Whether the event can be modified

busy

boolean

On shared or public calendars, whether to show this event's time block as available or not. (Also called "transparency" in some systems.)

recurrence

subobject

Included if the event is a master recurring event. See below for the subobject definition

master_event_id

string

Only included in exceptions (overrides) to [[recurring events]], the id of the recurring event.

original_start_time

unix timestamp

Only included in exceptions (overrides) to [[recurring events]], the start time of the recurring event.

 

What happens when a user leaves or disconnects their mailbox?

When a user is removed, or when they “disconnect” their mailbox, we :

  • Remove all synced threads and calendar events

  • revoke the mailbox token that needed to be able to establish a connection with the provider

Do I need to connect emails to send emails?

No. You can send and receive emails directly via Beamery (routed through our partner, Sendgrid), with replies tracked through beamery-generated reply addresses. To improve deliverability, this approach will typically involve setting up a ‘whitelabel’ for the domain, such that emails are sent via your domain. The process for this involves Beamery sending your IT team our email Cname. Your team then installs the cname, which will point to existing MX, SPF and DKIM records.

Is there a document which explains security on the Nylas side?

Yes - here is a link to the document

 


Please also review frequently asked questions in the page Beamery + Nylas - Connected Email FAQs