xTuple Connect Email Integration
Overview
Email integration provides a mechanism for users to import email into xTuple and have it be automatically associated with appropriate contacts and documents in the database. This solution allows users to continue using their preferred mail client and server software, but gives them control over what mail they want imported into xTuple.
The strategy to accomplish this involves two main components: The first part is a new open source email project called Trojita that is used as a tool to import mail into the xTuple database. Trojita is a lightweight open source IMAP E-mail client that can connect to any standard IMAP server. Trojita has been enhanced for xTuple to allow users to select personal IMAP folders to export to an xTuple database that has xTuple Connect installed. The synchronization from Trojita to xTuple is performed by Trojita in the background whenever xTuple is launched.
As mail is imported it is processed by business logic in xTuple Connect that associates recognized email addresses with their respective contacts. If a matching contact is not found, one is automatically created. Additionally, Connect scans the subject and body for predefined document reference identifiers and if any are found associates the mail with those documents. An Email tab has been added to the contacts window and other documents such as Sales Order, Incident and Opportunity so that users can quickly view email associated with those records. Mail launched from xTuple includes a document reference inserted into the subject line that can later be used as a reference to associate mail imported back into xTuple with that document. This reference should continue to work as users continue to volley email back and forth from their email client using standard reply mechanisms.
Finally, email sent by Connect itself using EDI profiles, such as sales order acknowledgements and incident updates, inserts an email record into xTuple as well. Note, however, it is not necessary to run the BatchManager processing utility to otherwise take advantage of mail integration.
You can watch a webinar recording of our presentation of Mail integration here.
Diagram of xTuple Mail Integration Components
Basic Installation and Configuration:
The following are the basic steps to installing and configuring Email Integration:
- Install or upgrade xTuple to version 3.6.1 or greater.
- Install or upgrade to xTuple Connect 3.6.0 or greater.
-
Copy Trojita files to any local machine where a user wants to perform mail synchronization
- The Trojita directory needs to be placed in the same directory as your xTuple ERP executable file.
- Launch the Trojita email client from xTuple and configure.
- Start the synchronization processor.
Upgrading xTuple
xTuple upgrades and release notes may be obtained from Source Forge in PostBooks downloads.
Commercial customers using xTuple Standard Edition may download upgrades from the xTuple Community home page in the Commercial Downloads section. Follow instructions provided for installation.
Upgrading xTuple Connect
xTuple Connect may be purchased from the xChange. Once purchased, downloads may be obtained form the xTuple Community home page in the Commercial Downloads section. Follow instructions provided for installation.
Copy Mail Integration files
When you uncompress the xtConnect folder downloaded from xTuple, you will find a sub folder called "trojita." Copy this sub folder so that it is in the same directory where the xTuple application resides. If you used the xTuple installer to install xTuple you should find the xTuple application in one of the following locations:
Windows: C:\Program Files\xTuple\Client\
Macintosh: \Applications\xTuple\Client
Linux: \Opt\xTuple\Client
Configure Trojita
Start xTuple and navigate to User Preferences. The Trojita email client it can be launched from the Email tab by clicking on the "Start Trojita Mail Client" button.
Once Trojita opens, click on "Settings" and select the "IMAP" tab. Enter your credentials to your IMAP mail server. Click Save.
You should see your IMAP mailbox folders appear in Trojita. Right click on the folder(s) you wish to import and select "Synchronize with xTuple." The folder should then change to have an orange arrow on it indicating it is a synchronized folder. When you are finished, close the Trojita Client
Once the Trojita configuration is complete check the "Synchronize Email while running application checkbox." This will make synchronization start when you start xTuple. You can click "Start Now" to start the process without having to restart xTuple. Click save and you are done with set up. You should be able to go to a contact in xTuple for which there was mail traffic in the folder you synchronized, and see mail history for that contact.
Install Notes for Microsoft users
Outlook
Mail messages opened from xTuple are processed as .eml files. If you are using Outlook 2003 or Outlook 2007, then you must apply the latest updates and certain registry changes so that Outlook will recognize .eml files. More information is available here. Note that Outlook 2010 supports .eml files natively.
Exchange
Exchange 2010 has IMAP services disabled by default. Read your installation instructions on Exchange to enable IMAP service. You can also start with online resources published by Microsoft here.
Install Notes for Macintosh Mail users
You may want to enhance your Macintosh Mail application to allow you to set up rules on outbound mail. You may purchase a utility for this called "Act-On" that may be downloaded for a 30 day trial here.
Install Notes for Thunderbird users
You may want to enhance your Thunderbird client to allow you to set up message filters on outbound mail. There is a free plug-in for this called "Send Filter" that is available here.
Viewing mail from xTuple
Email integration is supported on the following documents:
- Contact
- CRM Account
- Customer
- Vendor
- Employee
- Opportunity
- Project
- Quote
- Sales Order
- Transfer Order
- Purchase Order
- Return Authorization
- To Do Item
- Incident
Each of these windows has an Email tab from which you can view email associated with the document as pictured:
The plain text of the message may be viewed by clicking on a message in the list, or the entire message may be viewed by double clicking on the message which will open the message in the user's default email client.
The toolbar buttons operate as follows:
- New - Start a new message in the user's e-mail client with the To: address populated with the contact address on the document (if applicable) and an xTuple document reference in the subject.
- Print - Prints the selected email.
- Reply - Start a new message in the user's e-mail client with the To: address populated with the From address on the selected message. Subject will be pre-pended with RE: and the xTuple document reference if it does not already exist on the subject. Text from the body of the selected message will be added to the body of the reply message.
- Reply All - Same as reply except all addresses on the Cc column will be copied to the new message as well.
- Forward - Same as reply except prepend the subject with FW: and no addresses will be populated.
- Attach - Open an email attach dialog window that allows users to search messages by contact and associate them to the current document.
- Detach - Removes the association of the selected message from the document.
Messages launched from the email address link on the contact cluster also will include the document reference in the subject.
Attach Email Dialog
When a user chooses to attach a message an interactive dialog window allows users to search for messages by contact to attach to other documents. The contact cluster will allow users to select a specific contact, while a search box will allow them to further narrow their result set. The search box will search on address fields, subject and the body fields in the database.
When a message is selected and "Ok" is clicked an association will be made between the selected message and the document.
Does the xTuple client needs to be started to synchronize email in xTuple?
Will my email be synchronize while I'm in vacation, sick, ... ?
Yes, xTuple needs to be running for the Synchronization to work.
This is necessary because xTuple launches the synchronizer and communicates login credentials including database, user and password, so the it knows where to synchronize the data. We had originally thought about running the synchronizer as an independent service on the machine, but we subsequently decided we didn't want create potential security vulnerabilities by storing database credentials on the local machine which would be necessary to run it as a free standing service.
With the synchronization, if a user deletes an email from their email client, does that delete it from xTuple?
It does not.
Hi John-
Great webinar today. Thanks.
I set it up on my home laptop (Windows 7) and for some reason it immediately saw my email account on that laptop (even before I had configured Trojita).
So now, even thought I've got IMAP gmail account mapped, when I click new, it opens Outlook. What makes that even odder is that verizon.net, the email provider, does not provide IMAP.
More later,
George
I would just like to provide feedback to xTuple and others considering this functionality. Very, very well done!!
Despite this being a beta release xTuple have done a typically quality job in delivering the first release. I demonstrated the functionality to my Managing Director who was also suitably impressed by the depth of functionality. We really like the cross referencing between sender/receiver and account so the one email is visible from any of those sources. xTuple's email integration is already functionally superior to the two other commercial applications we have tried to use and the fact email now integrates directly into our ERP system and screens just takes this a world further.
I have worked for many years in the SAP field and not seen this sophistication.
I highly recommend this to anyone who has not purchased xtConnect already.
Cheers,
Dave
Hi,
How does the routine work to match documents?
Is there a convention? Project GREENLEAF...Sales Order 50205...
Was it cover in your presentation I couldn't attend!
Thanks,
When mail is imported into the database a trigger checks to see if a special reference is in the subject or body that has "xtref" and a document code in it. This reference is automatically appended to the subject of any email that is launched from the xTuple through your email client, and it is different for each document. Sales orders, for example, might be something like [xtref:S-012345]. This launching of your email client happens when you click on the email address of a contact or when you click "New" or "Reply" in the email tab of documents that have one.
Some have asked why we couldn't make the reference code hidden in an email header. The reason is because the mail is generated in your client by a "mailto" url mechanism. This is the same mechanism used to launch your email client when you click on an email address on a web page. The RFC for "mailto" (http://www.faqs.org/rfcs/rfc2368.html) does not allow for hidden information in such a message. The RFC states that for security reasons all information being sent must be clearly visible to the sender.
It is important to note that just because this reference is in the subject doesn't mean it will be imported. IT IS THE USER'S RESPONSIBILITY TO CONFIGURE THEIR MAIL CLIENT AND SYNCHRONIZATION SCHEME TO ENSURE THESE MESSAGES ARE IMPORTED, because these messages are ultimately being sent by your mail client, not xTuple. The reference tagging just helps xTuple sort out the relationships IF the mail gets imported.
The only messages that are guaranteed to be imported regardless of configuration are ones actually generated by xTuple Connect itself (NOT your email client) such as Sales Order Acknowledgements, Invoices, reports to run through xTuple Connect (AKA Batch Manager processor) or any mail generated via an EDI profile.
I know the differentiation between these two methods is bound to cause confusion, but we have thought long and hard about this and just can't think of any better way to handle these scenarios. That same mailto RFC precludes us from using the local client to send PDF attachments so we can't use the local email client for that purpose. On the other hand if we forced users to use our embedded email mechanisms for all email correspondence that would essentially ensure they don't use it at all because it just isn't realistic for us to build an email client with the rich user experience of those that have been under development for decades by large established organizations. I know from personal experience that end users simply will not tolerate anything less than what they are used to (i.e. Outlook, Mac Mail) when it comes to email clients. Not to mention the additional problem of accessing mail from multiple sources (web, phone, desktop). So this is how we arrived at the strategy we have implemented leveraging IMAP.
John
By the way, you can watch a recording of the presentation here: http://www.xtuple.org/node/3865
If two people running the sync client, get the same email, will it appear twice in xtuple?Example is if the customer includes two precipitants. If so, what is the duplication identifier based on? Thanks
They won't. Both the synchronizer and xTuple connect processors calculate a cryptographic hash on the message body that creates a unique ID that is stored with the message in the database. They compare to see if there are any messages with the same hash ID already in there before attempting to insert a new copy. If one is already found, it will skip that one.
It appears that on a windows machine the configuration parameters for the trojita client are stored in the registry. It also appears that only one set of parameters can be stored. It seems that would prevent a user from having this functionality on multiple databases at the same time. In other words if an xTuple customer actually has multiple companies and we know some do as we have put some effort in combined financial statements, etc, a user would only be able to import into one database from a machine. Is that correct?
That is correct.
Rather than setting up email filtering on an individual mailbox basis, will it have the same effect if you setup another email account (with xTuple/Trojita running continuously) to accept all incoming emails and archive to xTuple? This way, it's easier for a person to just CC the message, instead of making sure the message is located in the correct folder? The question is will the tool recognize the sender, recipient, etc. the same way and reference the message correctly for archival?
Also, can you place the xtref tag inside the body message (maybe at the end?) instead of in the subject line?
Yes and Yes.
