Security Scanning Certificate Testing

[maksum]

I've always wondered how effective these security scanning services are. According to the testimonials on their web sites they are VERY effective in increasing consumer confidence and increasing conversion rates :D (surprised?) I've wondered if they just really aren't that necessary, since an inquiring customer can see that we are PCI-compliant by reading through some of our pages.

However, we've actually had more than one customer phone in their order and tell us they won't place the order through our site because we don't have the daily scan deal on there. One of them told us that she had her credit card stolen and as she was resolving things with her credit card company they were giving her some advise and told her to never shop online from a site that doesn't have that.

So I decided to become a bit serious with it and I've been doing some research on these services, such as Trust-Guard and McAfee (which interestingly enough is pronounced MACafee instead of McAfee according to their telephone system).

Since I've written a request form to McAfee and never got a response, AND called them and was put on hold for 10 minutes without a response I decided to try working with Trust-Guard. My thinking is if they provide the same service, I don't think my customers care which specific brand is there.

So I'm all signed up with Trust-Guard and after making a couple of tweaks to my site I'm passing their scans and I have the seal prominently placed on my web site (feel free to check it out at www.KarensCookies.net ). I have also started what is called "A-B Split Testing" to see how well it's going to work.

Basically what this does, is it shows the certificate to every other customer. They record conversions (visitors turned buyers) for both scenarios (certificate seen vs unseen) and post the results.

SOOOOOO.... the point of this big ol' post??

I'm just letting anyone interested know that I'm doing this and wish to share the results of that testing. Perhaps the information, and my ultimate decision to pay for the service or not (the daily scan service is $67 monthly), will be helpful to someone else considering the use of such services.

I know that it's been brought up before, with some light discussion, but I wanted to provide some concrete data from my own experience with a 3-D Cart Store.

Ok, so here we go. A-B Split Testing is starting today (Feb 13, '10). My conversion rates for the last 30 days has averaged 2.52% before the implementation of the seal.

I'll keep you posted....

[Mark]

Sorry, can't see the seal. :D
I must be an "A" customer, err....or a B customer.......
I suppose it caches my IP or places a "cookie" heh heh on my PC. :D

[cbsteven]

Yeah I am curious about how you implemented the A/B testing

[maksum]

Yeah I am curious about how you implemented the A/B testing

It's a feature that trust-guard offers. In fact, they have a 60-day money back guarantee and in order to qualify I think you have to do this testing to show that the seal didn't make a difference.

Basically it's a little script that calls in the graphical seal. So they can control whether or not it shows. they have you put a little snippet of code on the thank-you page after checkout. Between the two of those they can tell when a sale has been made and whether or not the seal was showing at the time.

I'm glad you told me you can't see it Mark, as I was refreshing and couldn't get it to go away. Perhaps the IP is recorded or something.

Mike

[maksum]

A quick update on time-line... They will begin providing me with analyzed data when each version of my test has at least 1,000 impressions. I'm at 195 total impressions since last night, so in approximately 5-6 days I might be seeing some information.

[maksum]

Ok, I reached over 2,000 impressions and starting to see some data. The test is only about half-way through, but I thought I'd share what I have so far...

take a look:
http://forums.3dcart.com/attachment....1&d=1266597823

Seals Off Conversion: 3.89%
Seals On Conversion: 3.02%

So yeah, so far it doesn't appear to have helped :) There isn't a HUGE gap between the total # of conversions, so I'm not drawing any conclusions yet. We'll see what happens when the remainder of the tests are complete.

Mike

[maksum]

Here's the latest update. The test is not done, but has been running for a few days and has seen over 120 sales. So far it seems that it is not making a difference. One might argue that sales are worse with the seal (which is technically true at this point), but the gap is so small I'd say it's just arbitrary.

Take a look at the chart if interested.

http://forums.3dcart.com/attachment....1&d=1266945512

So does anyone else use these and see an improvement? Has anyone else tried them and saw no difference?

I'll keep testing, but so far it wouldn't pay for itself :)

[Mark]

wow.......
I guess you get a freebie! :)

Have they contacted you and grovelled yet? :D

[maksum]

Yeah, the gap was widened just a bit since posting that too... still in favor of no seal. Right now the score is 73 to 68. I can't imagine the seal actually turning someone away, so I think it's just chance that no-seal is winning out.

If it keeps up I will hit them up for my guaranteed refund and stop using it I suppose. No need to throw money away.

[Mark]

In today's day and age, I'm not convinced the paid seals really offer any more than other "seals". For example, there are eight icons or images that reference "security" on your home page. And admiditably, we have several as well. Personally, I think the limit should be two. more than that should be avoided. I may take another look at our own site as well. :o
To be honest, I would think a telephone number would bring more conversion than a security seal. And they cost a lot less. :)

[GoldenEagleOutdoors]

I also think that an abundance of "security" seals are a waste of effort. I don't think people really care about them all that much, maybe because they don't understand what they are all about. The only seal that I think may have a positive effect is the BBB seal since they are a well known organization. I have noticed that the larger ecommerce stores don't have any, or they only have one or two.

[maksum]

Well, maybe people want to see just SOME security deal. Maybe I am not doing any better because when the seal is off they see the other stuff. A couple of them I just whipped up myself in photoshop :)

At any rate it's been interesting. I wonder how things would change if that were they ONLY seal showing, and half my visitors saw nothing.

Hmmm, might be worth investigating... but even if it came back with an increase in conversions with the seal I wouldn't pay for it, knowing it doesn't matter what I put on there.

[Mark]

how's the test coming along. I see you still have the seal.....

[maksum]

how's the test coming along. I see you still have the seal.....

Still testing... 159 conversions for Seals Off, and 145 for Seals On. I called them up a couple of days ago and they suggested I remove a couple of the seals from the shopping cart and check-out pages. Before I had all 4 of their seals on there. Now I just have the "security scanned" and "privacy verified" seals.

I doubt it will make a difference. I'll let it run for a couple more weeks, just so I can be confident that I gave it a fair shake, and then likely end it and ask for my money back.

As I mentioned, I really think people like to see some sort of security seal action going on, but my hunch is that they don't care specifically what it really is, or if it clicks to a 3rd party scanning service with a report. When the seals are off they can see my other "seals".

If I had NO other "seal" and was running the test, then I might see more of a difference, in favor of the seal... but at the end of the day, that would perhaps just be testing the effectiveness of "seals" in general.

Mike

[maksum]

Quick update,

Not much has changed since I reduced the number of their seals I had on my checkout pages. It seams to have reduced the gap a bit and essentially made it so seals-on and seals-off are closer to equal now. Certainly hasn't made the seals-on scenario better though.

Right now the score is:
Seals Off: 217 Conversions (4.35%)
Seals On: 203 Conversions (4.07%)

See chart here:
http://forums.3dcart.com/attachment....1&d=1268758700

I called them up to start the process of canceling my account and getting my money back, but they said I'd need to talk to the manager who wasn't in at the moment. He's going to give me a call back.

I'll chat with him just to make sure I'm not missing something, but it seems clear that it's not improving my sales. I could possibly be convinced to run the test longer, but I doubt it.

Mike