Marketing setup journey (part 1 of 3) - EXM

This blog will present a series of articles about my journey on implementing custom Marketing capabilities.
While Sitecore offers a lot of capabilities in their platform, to make use of them in your custom implementation there are quiet some steps to take and code to write to fulfill your requirements.
My journey concerns setting up an environment with Email Experience Manager (EXM) with a custom Mail sending api, engagement plans, and exprience profile extensions.
Requirements I had to fill were:

• Based on custom contact data (like age category, nr of acquired products etc.) enroll contact in engagement plans and send them automatic email campaign messages and track their progress on the site after opening the email.

This sounds like a simple enough requirement. But implementing it wasn't as straightforward as this one liner. In my coming blogs I will show you what I needed to do to enable this kind of implementation.

This first blog will focus on setting up EXM and a custom dispatch provider to hook into the message sending and use a custom API for this instead of the default available Sitecore cloud services and SMTP implementations.

My environment consists of a Sitecore 8.2 update 5 default installation when I start my journey, with a local mongodb and local SQL Express on my development machine.

Prepping the environment

To setup my development environment including EXM , I had some attention points during setting it up, which were not entirely clear for me coming from the Sitecore installation guide for EXM, so therefore I will post here some duplicate steps, but hopefully with a little extra info here and there to help you in a smooth setup.
For Sitecore 8.2 update 5, you need to download EXM 3.5 from the Sitecore developer site.

1. Download Sitecore 8.2u5 zipfile and install to desired location using SIM 
2. Install mongodb:
• Download it here
• Execute install wizard
• Launch mongodb (always after booting your machine by starting from commandprompt: mongod.exe (located somewhere like: C:\Program Files\MongoDB\Server\3.4\bin) or create a satrtup script or windows service to to this for you automatically
3. Install something like mongobooster to easily browse through your mongodb (this will come in handy later on when we are extending the experience profile)
4. Download EXM 3.5 and install according to installation instructions from Sitecore
• Important to note here, is that it is absolutely necessary to have the empty connection strings in place, otherwise installation will fail. 
5. For my setup with a custom API to send the emails, I used the Sitecore.EDS.Providers.CustomSMTP.config and Sitecore.EDS.Providers.CustomSMTP.synced.config files which are located in App_Config\Include\EmailExperience, so I removed the .disabled from them and customized them:
• In _CustomSMTP.config I changed the email dispatch provider node

                <!-- EMAIL DISPATCH PROVIDER
                    Provides email dispatching.
                -->
                <dispatchProvider defaultProvider="default">
                    <providers>
                        <clear />
                        <add name="default" type="Website.CustomDispatchProvider, Website">
                            <param ref="exm/eds/connectionPool" />
                            <param ref="exm/eds/environmentIdentifier" />
                        </add>
                    </providers>
                </dispatchProvider>

• With this change in my 'Website.CustomDispatchProvider' class I implemented my custom code, see more info below

Configuring your email campaign message

Now that EXM has been setup it is time to create your first automated email campaign message.

1. Login as admin, go to dashboard and click Email Experience Manager
2. Click ‘Default settings’ on the top right corner, in here set the following values:
• Base URL: to your domain name, this is used to prefix any link to a local Sitecore item you will insert in your emails, so that these work from an email client.
• From email: configure here the mail address that you want to use as a default, as sender address
• Other fields are optional so you can leave them empty for now
3. Now click the Create button in top left corner and choose menu option: ‘Automated email campaign’. Select ‘Simple HTML message’ and give it a proper name (for example: MotivateToSetGoalsMail) and click create.
4. A wizard appears
• On general tab: You can select campaigns here, although it is not necessary, it is recommended to create a campaign in Sitecore (under marketing control panel / campaigns) so that the email is categorized under it and all analytics regarding the email are gathered there. This is convenient for reporting and dashboarding.
• Message: Here you set a subject for your email. For example: ‘Set the goals for your assets now to make optimal use of our environment!’
  Set a body, for best results on analytics also include a link to a page on your site, to see better results in the email campaign and get data into your tracking reports
• Review: Send a test mail to your own email address to check the mail to see if the link has been properly prefixed with your hostname, and see in the source of the mail that a img tag/ pixel has been added to track the email. If you have the code for your custom dispatcher in place (as described below) you will be able to set a breakpoint and debug to double check your code is executed.
• Deliver: Select Activate message now, and click Activate message.
  Your automated campaign is now active!

Custom dispatch provider

As described, for my scenario I needed to have a custom (CRM) api as the sender of the emails, so that these emails could be archived in CRM for this contact. To do this I needed to create my own dispatch provider. (Thank you Maarten Willebrands for your help on this, the Sitecore community and many blogs out there, helped me many times in my journey!)
As previously described, in the custom SMTP config I set it to use my custom code instead of the default Sitecore one. The code for this custom provider looks like this:

   public class CustomDispatchProvider : DispatchProviderBase
    {
        private readonly IChilkatConnectionPool connectionPool;
        private int Size { get; set; }

        public CustomDispatchProvider(IChilkatConnectionPool connectionPool, IEnvironmentId environmentIdentifier) : base(environmentIdentifier)
  {
            Assert.ArgumentNotNull(connectionPool, "connectionPool");
            Assert.ArgumentNotNull(environmentIdentifier, "environmentIdentifier");
            this.connectionPool = connectionPool;
        }

        public override async Task ValidateDispatchAsync()
        {
            return await (await this.connectionPool.GetConnectionAsync()).ValidateSmtpConnection();
        }

        protected override async Task SendEmailAsync(EmailMessage message)
        {
            Stopwatch stopwatch = Stopwatch.StartNew();
            //ITransportClient transport = await this.connectionPool.GetConnectionAsync();
            
            var mailMsg = new System.Net.Mail.MailMessage { From = new System.Net.Mail.MailAddress(message.FromAddress), Subject = message.Subject, IsBodyHtml = true, Body = message.HtmlBody.Replace("your.host" , "prospery2.local")+ message.CampaignId };
            mailMsg.To.Add(message.Recipients.First());
            MainUtil.SendMailAsync(mailMsg); //Replace with your call to the API or test with this version to realy with your default smtp provider
            this.Size = message.HtmlBody.Length;
            stopwatch.Stop();
            var result = new DispatchResult
            {
                Statistics =
                {
                    {
                        "SendingTime",
                        stopwatch.ElapsedMilliseconds.ToString()
                    },
                    {
                        "Size",
                        this.Size.ToString(CultureInfo.InvariantCulture)
                    },
                    {
                        "ParsingTime",
                        stopwatch.ElapsedMilliseconds.ToString()
                    },
                    {
                        "ConnectionTime",
                        stopwatch.ElapsedMilliseconds.ToString()
                    }
                }
            };

            return result;
        }

        protected override void SetMessageHeaders(EmailMessage message)
        {
            base.SetMessageHeaders(message);
            message.Headers.Set("X-Sitecore-Campaign", message.CampaignId);
        }
    }

It is very important to add the DispatchResult with all the 4 Statistics or your Send Email pipeline will fail. Your email will be send, but the pipeline will not remove your email from the Dispatch Queue and after an hour your email will be send again and again.

That's it for the first blog of this series.
I now have described how to setup your environment with EXM and how to create your first automated message, dispatched by a custom provider.
Next step in this series will be to extend the experience profile of your visitors and after that, an engagement plan that based on custom experience profile data coming available triggers an EXM message and following it's actual engagement value.