Threadloom Blog

Curating community content.

Research

Primary Login beta results

·

Last month, we shared about a new feature we’ve been working on called Primary Login.  Primary Login lets forum users securely register and/or log in to a forum without a password.

We’ve been beta testing Primary Login on The Admin Zone and several partner sites, and are excited to share the initial results.  In summary, Primary Login has reactivated lapsed forum users and encouraged users to post more often to the forum. 

Experiment setup

In our test, we added call-to-action (CTA) buttons to each forum’s email newsletters.  The CTA buttons directed users to enter their email address and click on a link in a confirmation email.  If the email address was already associated with a forum account, Primary automatically logged the user in; otherwise, it directed the user to the forum account registration page.

Primary Login CTA button

Results

Over two weeks, we sent email newsletters with Primary Login CTA buttons to over 100,000 recipients.  The conversion rate (i.e., of users who clicked on the CTA button, the percent that actually logged in) was 55%.

We also saw increased behavior on the forum.  Comparing the two-week periods immediately before and after launching the experiment, posts by Primary Login users increased 33%.  After excluding a handful of frequent posters, the figure increased to 39%. 

Of these users, one-third had not created a post in the prior two weeks, and half of those users had not created a post within the prior six months.  In other words, a significant number of users were being activated or re-activated as content creators.

Finally, we compared post rates of Primary Login users vs. regular (i.e., non-Primary Login) forum users.  Sixteen percent of Primary Login users posted in the two-week period after launch, compared to 6% for regular forum users.  Primary Login posters averaged 1.59 posts per user vs. 0.44 posts per regular forum user.

Posts per user for Primary Login users vs. regular forum users

What’s next
We’re looking forward to sharing where Primary Login is going next, and how this will work with where we are going with Product Stories.  In the meantime, if you’re interested in learning more about Primary Login, feel free to drop us a line.

Product Stories: The second experiment

·

In last week’s blog post, I shared about an experiment that we ran last year with AZBilliards, Home Theater Forum, TalkBass, and Windsor Peak Press.  To recap, we created a new site called Product Stories with a product-focused presentation of forum discussions.  The initial results were promising, but were left with two questions:

  • What if we made the products the focal point, instead of the original post?
  • What if instead of including full post text, we only displayed a post snippet and forwarded clicks to the original forum thread?

To answer these questions, we designed a second experiment with Tesla Motors Club, Tortoise Forum, and WDWMAGIC.  First, we highlighted a specific product, then attached related posts from various forum threads.  Second, we included a brief snippet that linked back to the original forum post.

We also explained to visitors how the affiliate model on the site worked.

As with our first experiment, we generated traffic through Facebook ads, and again targeted non-forum, passive users.  

The overall engagement rate was 23%, softer than our first experiment.  Amongst those sessions, however, there were three interesting results:

  1. The purchase conversion rate improved to 3%, compared to 1% in our first experiment.
  2. Participation was more action-oriented.  In the first experiment, 53% of engaged sessions clicked on a post or product.  In this experiment, that figure was 76%.
  3. About half of these clicks went directly to the original forum post, while the rest went to the retail affiliate link.  The percentage going to the original forum post was much higher than we expected, especially considering that the retail link included a large image above the posts, and a big button below the posts.

The results convinced us that there was a compelling model here, and that Product Stories was worth building out.  But it also gave us insight into something broader.

When we saw that half of the Product Stories clicks went back to the forum, we put those results alongside the retention figures from the first experiment.  We realized that with some elbow grease and luck, we might be able to create a model with a positive click multiplier for forums.  In other words, a single session from Product Stories could, over time, produce more than one click to the forum.

We also noticed something else in the retention data.  We had run a series of early experiments in parallel alongside this Product Stories experiment, testing various presentations of non-product forum stories.  It turned out that users who visited both non-product forum stories and Product Stories were much more likely to return than users who visited just one or the other.

That led us to our second takeaway: By extending the Product Stories model to non-product forum stories, we might be able to further multiply the number of clicks referred to forums.  In other words, it might be possible to finally convert passive drive-by visitors into repeat visitors.

Since wrapping up this second experiment, we’ve run a number of additional experiments to test this hypothesis.  We still have much more work to do, but I’m happy to share that in partnership with our customers, we are getting closer to the goal, and I hope to share more about those results soon.

Product Stories: Exploring new ways to present forum content

·

Beyond our Search and Newsletter products, we are continually exploring how to better connect non-forum users with our customers’ communities.  Over the past year, we’ve partnered with several customers to test new ways of presenting forum content to increase engagement while also generating revenue.  Since nearly every forum includes discussions about products, we decided to see if we could make these stories more relevant to people outside of these communities.

For our first experiment, we partnered with AZBilliards, Home Theater Forum, TalkBass, and Windsor Peak Press.  Using a set of recent threads, we extracted canonical products, linked them to retailers, and grouped posts around each product.  This was a one-time effort (in other words, we didn’t update products or posts at all during the experiment).

We set up a site called Product Stories and generated traffic through Facebook ads.  We targeted non-forum, passive users, similar to what a forum might see from organic search.  Over the course of one week, we sent about 9K sessions to a product story on the site.

About 13% of sessions clicked on a product link, of which, about 1% purchased a product (Fig. 1).  The click-through rate was higher than we had targeted, but the conversion rate was lower. So an ok set of results – not stellar, not terrible.

However, we were much more interested in engagement and retention.  A low purchase conversion rate paired with high engagement and retention rates is more sustainable than the inverse.  Fortunately, the Product Stories engagement numbers were more interesting.  The bounce rate was 69% and average time on page was 4:22, both much higher than we had targeted.

Fig. 1: Percentage of sessions that engaged with the site

Exploring a Product story38%
Expanding a forum post40%
Clicking on a product link13%

Fig. 2: Google Analytics overview

More importantly, retention was significantly higher than average forum retention rates, both in the aggregate and at the individual forum level.

Fig. 3: Google Analytics retention data

The initial Product Stories results were encouraging, but also raised more questions:

  • What would happen if we promoted the products as the focal point (instead of the original post)?
  • In this experiment, we had included the full post text on https://threadloom.shop.  What would happen if we only displayed a snippet, and forwarded “more” clicks to the original forum thread?

In my next post, I’ll talk about our second Product Stories experiment, and what we learned.

How to be a happy car owner in 2017

·

Last year I inherited a minivan from my brother. I’m barely 5 feet tall, and I felt like I was driving a cruise ship. A few passenger side dings and scrapes later, I started researching how to buy a new car. I narrowed down my choices to a Honda CR-V, Toyota RAV4, and Subaru Outback. I spoke with friends who owned those cars, did numerous test drives, and checked out car maker websites and a few blogs.Read more

Subscribe to the blog

Sign up to receive new blog posts by email.