How email split testing actually works

Re: email testing Following is from BerserkerMail creator Troy Broussard who had to do “for real” science as a nuclear engineer in the Navy, and then later as Executive Director of Technology at Encyclopedia Britannica running a $12 million department of developers... then took that rigorous discipline & knowledge of how actual testing works to his clients doing the email automation/testing/tracking for as many as 30 - 50 million emails per month at times for them. Here’s what he wrote in the BerserkerMail Mead Hall Facebook group: === These other [email] platforms have received 100's of millions of investment dollars and still are just trying to "out-feature" each other in a spiraling race to 1001 features they can list on their comparison pages all of which provide no value or real benefit to their users... A perfect case in point is how the majority of them approach split testing. Now first off, most people DO NOT NEED to do split testing, but many wanted it, so we added it into BerserkerMail even though neither Ben nor I use it. But most platforms just randomly pick 50% and send subject line A and send subject line B to the other 50% and call that a split test. EPIC FAIL For a split test to be even remotely useful, each of the two splits must be mostly similar in terms of email ESP distribution (number of gmail contacts, number of yahoo contacts, etc) as well as mostly similar in terms of account age (did they just join your list yesterday? or 3 years ago? deliverability will vary dramatically for these variances). So if we're gonna do something, we do it right. When you do an email split test in BerserkerMail, we allow you to control the algo we use for the split — by email ESP domain, by age, or a combination of both (recommended). Again, split testing is not for most people and you have to have a fairly large list for it to even make any sense, which is why I don't personally use it. But I do feel it is worth noting HOW we do it and the CARE in which we take in our approach to making it meaningful to your business if you choose to use it... === It got me to thinking about something else reading that too. And that is the difference between having lots of Features vs lots of functionality. Rarely the twain shall meet in SaaS — which is probably one big reason why so many people wince these days when their favorite software is updated or they're charged for features they never use. I remember BB (before BerserkerMail) I’d constantly wince whenever one of the platforms I used did an update. It always — almost without exception — made things more complicated. Made things more time consuming, too. And it also required more screens navigated, boxes ticked, needless clicks needed just to load, schedule, and send a friggin’ email. And that’s not even going into the campaign building, tracking, and testing sides of things. From what I hear that’s only gotten worse with a lot of ESPs. And who do we hear this from? From some of those fleeing the various pronoun platforms. We are a platform run BY email marketers - and Email Players subscribers, I will add - who sell with email themselves — from the owners to the COO to the attorney we hire to keep us compliant and even to our podcast producer. If you want to see if we're for your business or not, simply get a free test drive. You'll also get access to our mobile app with training inside. As well an eBook I wrote about how to use email to prosper even during a worldwide financial crisis. Here is the link: [https∶//www.EmailPlayers.com/berserker]( Ben Settle This email was sent by Ben Settle as owner of Settle, LLC. Copyright © 2024 Settle, LLC. All Rights Reserved. No part of this email may be reproduced or transmitted in any form or by any means without written permission from Settle, LLC. Click here to [unsubscribe]( Settle, LLC PO Box 1056 Gold Beach Oregon 97444 USA